Atlassian Plugin Software Testing and Quality Assurance Services

Atlassian team

As a team of accredited Atlassian plugin developers, we are highly thorough when it comes to ensuring the quality of our projects. But when it comes to novice programmers, it is common for them to either skip plugin testing completely or to have no framework for handling it as efficiently as possible.

Importance of Atlassian Plugin Testing

Atlassian plugins are a solid way to power up your workplace, manage teams efficiently, and complete projects faster. Tools like Draw.io for data visualization or Scriptrunner for creating and updating custom scripts are extremely helpful for teams. 

However, Atlassian plugins can be a huge pain in the neck as well. Some tank your workplace’s performance, others require constant updates or have a steep learning curve. 

That’s why Atlassian’s official recommendation is to keep the workplace lean, using no more than 5 plugins at once. 

With the competition so fierce, plugin developers need to make sure that their product is functional, easy to use, and non-straining in performance before releasing it. That’s why programmers can’t afford to skip QA testing. 

Here are the top reasons to test Atlassian plugins:

• To make sure it works well in all browsers and supports mobile devices; 
• To validate reliability and avoid security threats (buffer overruns, injections, and other vulnerabilities); 
• To ensure that the plugin is easy to configure and use; 
• To validate the plugin’s compatibility with other products in the marketplace and the latest version of Atlassian tools (we’ve seen plenty of cases of a single bug making a plugin fully incompatible with others); 
• To ensure high performance — slow Atlassian plugins hinder teams’ productivity so developers remove them shortly after installation.

Objectives of Atlassian Plugin Testing

Quality assurance teams collaborate side-by-side with development teams, contribute to best coding practices, and make sure plugin software engineers take into account the following characteristics. 

1. Functional quality

Working alongside software testers, the development team should be able to ensure that the finished plugin works under extreme conditions — low network signal, poor hardware, high traffic, and others. Quality assurance allows the tech team to catch new bugs and functional errors, and spot these before the release of the final build. 

2. Development speed

Atlassian plugin testing teams should aim to help developers maximize their efficiency and allow them to ship features into production faster. QA engineers help improve the speed of development by outlining best coding practices, automating unit testing, and writing clear documentation that’s easy to use for references. Some also facilitate the process through test management for Jira.

3. Risk handling

When designing and executing cases for plugin testing, quality assurance teams should aim at not only flagging existing threats but also reducing the risks of new functional, security, and performance bottlenecks that may appear after the release. Collaborating with developers to create contingency plans and detect potential errors saves tech teams and business owners a lot of time, money, and effort. It also contributes to high-quality user experience.

4. Compliance with Atlassian requirements

To host their products on the Atlassian marketplace, tech teams need to make sure that these tools meet a set of requirements: 

• Have a functional, easy-to-use UI. Atlassian moderators pay a lot of attention to testing interfaces and sorting out unfunctional ones; 
• Offer robust, well-formatted documentation. Development teams need to work out clear guidelines on using and configuring the app. Having well-documented bug reports also increases your odds of passing app moderation successfully; 
• Perform well. The app shouldn’t slow down the performance of Atlassian tools or impair their functionality; 
• Ensure data safety and protection against third-party attacks. The app shouldn’t create breaches that would lead to the disruption of Atlassian’s native security. 

QA managers ensure that Atlassian plugins meet all functional and non-functional requirements to be hosted on the marketplace. 

Types of Atlassian Plugin Testing

Atlassian plugin testing can be broadly divided into two categories — functional and non-functional. 

Functional testing validates the features of a plugin, ensuring all activities lead to the result defined by the development team. 

Non-functional testing, on the other hand, assesses whether or not the system is resource-intensive, how well the interface is designed, whether or not end users enjoy interacting with it, etc. 

Atlassian Plugin Functional Testing

There are several processes our development and QA specialists use to validate the functionality of an add-on. Let’s take a look at the most common functional testing activities for Atlassian plugins. 

Unit testing

Unit tests are the most basic testing activity usually handled by development teams. These bite-sized tests help tech professionals ensure that every code module meets functional requirements. 

Here are the objectives of unit testing:

• Flagging and fixing bugs before a feature is shipped into production and connected to other modules; 
• Testing all plugin codes in small chunks autonomously as soon as a developer finished working on it; 
• Improving the interdependence of code fragments and increasing the overall product performance. 

API testing

Developers often integrate REST API into their projects. Ensuring the reliability of API communication is a crucial stage of add-on testing. Unlike UI tests that often need one or a few rewrites at the development stage, API testing is more immutable and controllable. That’s why they are easier for QA teams to handle. 

Here are the objectives of API testing:

• Validating data accuracy, input formats, error and status codes; 
• Pinpointing and eliminating server-side bugs; 
• Ensuring that the plugin is still functional even if its production environment deviates from its ideal conditions. 

Installation testing

Atlassian QA teams also perform installation testing since it ensures that there are no bottlenecks in the configuration process. 

Here are the objectives of installation testing:

• Determining the prerequisites for installing the plugin (RAM availability, browser version, and others); 
• Ensuring that a user can install the plugin even if the network quality is poor; 
• Making sure that all plugin files are successfully removed from the workplace after a user uninstalled the files.

Regression testing

This stage of plugin testing helps QA teams determine that an update doesn’t get in the way of add-on’s other features. Typically, a regression testing is a re-execution of previously written test cases after an update to make sure that the overall functionality of the plugin is unaffected. 

Here are the objectives of regression testing:

• Ensuring the plugin’s stability after an update; 
• Validating the efficiency of bug fixing; 
• Checking that a part of the tool doesn’t interfere with the functionality of other components.

Atlassian Plugin Non-Functional Testing

When it comes to non-functional testing, our team focuses on making the plugin as easy to use, lightweight, and reliable as possible before it hits the market. There are several stages we include in the non-functional testing roadmap.

Performance testing

Performance testing is one of the crucial non-functional testing stages. It consists of several testing activities and helps plugin developer teams ensure that the add-on is fast and capable of supporting high user loads. 

Here are the objectives of performance testing:

• Assessing the plugin’s performance speed; 
• Validating the resource consumption and finding ways to optimize it; 
• Determining the number of concurrent users that the add-on can handle without crashing. 

Security testing

Since add-ons are connected to teams’ workplaces, they can be exploited by third parties as gateways to sensitive data. To make sure no hacker can penetrate the defense line of Atlassian tools, the development team thoroughly assesses the security of plugins before hosting them on the marketplace. 

To make sure your project meets all the guidelines, consider running security testing before sending a plugin for moderation. 

Here are the objectives of security testing:

• Making sure there are no exploits that could lead to data loss or allow hackers to manipulate the workplace on the team’s behalf; 
• Exposing the vulnerabilities of the solution and creating a contingency plan to eliminate security threats; 
• Improving the plugin’s protection against intrusion with the help of multiple-step authentication, encryption, and other measures. 

Documentation testing

Documentation testing is an essential part of Atlassian plugin quality assurance. Its goal is to prove that the way in which documentation describes the tool’s setup process and features aligns with the system’s actual functionality. 

Here are the objectives of documentation testing:

• Making sure that all the features mentioned in the documentation are available in the latest add-on version; 
• Ensuring that the setup and configuration processes are explained step-by-step and no crucial elements are out of the spotlight; 
• Avoiding generalizations and misleading statements that will increase the number of support calls, queries, and negative reviews. 

Recovery testing

To make sure that an Atlassian plugin is capable of withstanding threats and recovering from crashes, tech teams create robust disaster recovery plans (or DRPs). It’s a tester’s job to check these contingency strategies at the stage of quality assurance known as recovery testing. 

Here are the main objectives of recovery testing:

• Determining points of failure and assessing the system’s ability to recover from crashes and damage; 
• Ensuring that the disaster recovery plan is capable of fully bringing the system back to its pre-failure condition;
• Validating the integrity of user data after the plugin’s crash or in the environment of poor connectivity.

Process of Atlassian Plugin Testing

If you are about to release an Atlassian plugin, you might be wondering how to structure QA activities. In general, the quality assurance roadmap varies depending on the team size, the scale of the project, and the deadline constraints developers might be bound by. 

However, our team outlined the following must-have processes that contribute to successful Atlassian plugin testing. 

1. Gathering testing requirements

Before you start testing, it’s important to be clear on the objectives that the plugin should meet. Our team divides all software testing requirements into three categories:

• Functional requirements that cover the way the add-on’s features work; 
• Non-functional requirements that focus on performance, scalability, reliability, and security; 
• Business objectives that validate the plugin’s ability to generate brand awareness and revenue for the company. 

We gather testing requirements by following these practices:

• Discussing the vision and ambition behind the project with its stakeholders; 
• Prioritizing critical features and metrics over lower-tier ones; 
• Updating the requirement list regularly throughout testing (new objectives often resurface after a deeper look at the program); 
• Making sure that technical requirements don’t contradict business objectives. 

2. Generating test cases

Once all stakeholders and testing teams have negotiated functional, non-functional, and business requirements for the product, we move on to the next stage — writing Jira test cases. 

A test case includes a collection of data inputs, a description of the environment, and the estimated results of the assessment. 

The process of test case generation varies from team to team — here’s how we handle it. 

First off, we design functionality-focused Jira test cases that validate the plugin’s basic features:

• Positive tests: entering inputs that comply with the feature’s conditions (e.g. numbers in a number-only data box);
• Negative tests: entering inputs that contradict the feature’s conditions (e.g. letters in a number-only data box).

3. Selecting representative test cases

At this stage, our QA testers choose the cases that would demonstrate the state of the system to the greatest extent. Knowing how to select representative cases helps testers save time during QA processes and prioritize first-tier insights over less impactful ones. 

Here are a few examples of representative cases for plugin testing:

• Checking authentication results (both scenarios in which users enter valid and invalid credentials);
• Checking the results of the installation process; 
• Validating the behavior of the plugin when a user chooses “Uninstall”; 
• Making sure all the user data is saved after a system crash.

4. Executing test cases

This stage of quality assurance is what most people understand as software testing — executing pre-selected cases. 

Before proceeding to this activity, our team configures a testing environment where the test will be executed. There are two components of a testing environment — hardware (physical devices) and software (IDE, test automation, and other tools). 

After configuration, QA professionals execute plugin tests and conduct test management for Jira. Here are the practices we follow to improve the efficiency of test execution:

• Thoroughly documenting test results so that they are easier to analyze later; 
• Tracking defects automatically and making sure we get notifications as soon as an error is detected; 
• Ensuring the security of our tests by versioning them, keeping off-site backups, and monitoring access control; 
• Ensuring full test coverage to not miss out on validating an important feature. 

5. Analyzing the results

Once testing documents and reports are ready, the team analyzes them. Based on the insights gathered by testers, project stakeholders decide on whether or not the product meets functional and non-functional requirements. 

Here’s how we determine the validation of requirements:

• Running impact analysis; 
• Using requirement management tools to coordinate the process;
• Conducting time-sheet analysis; 
• Ensuring all requirements have been tested. 

6. Passing the results over to the development team

Getting in touch with the development team and discussing testing results should be the final stage of software testing. Along with testing documentation, our QA professionals inform software engineers about the risks associated with the plugin, best practices, and guidelines.

Conclusion

Atlassian plugins are powerful tools that help teams all over the world improve their productivity. However, it’s up to the team behind the product to ensure that an add-on doesn’t stand in the way of users’ efficiency. That’s why it’s important to test the product thoroughly before launching it on the marketplace.

To get first-class Atlassian plugin development and testing assistance from the company’s certified partners, contact Rozdoum. Our QA specialists employ the repeated testing methodology to make sure your plugin fits both functional and non-functional user  expectations. 

To get in touch with us — leave a message. A project manager will get back to you to discuss the project in a couple of hours.