01Test Planning_
Trainee
Junior
Middle
Senior
Expert
Trainee
- Has basic theoretical knowledge of what estimation and prioritization are: able to estimate simple tasks and explain them
- Requests task priorities from N+1
- Knows types of test documents and their purpose
- Escalates in case of blockers or wrong estimations
Junior
- Has practical testing experience and understanding of the testing process
- Can read and understand the current test plan (their own activities and the current approach) with some guidelines
- Able to estimate non-complex tasks with an acceptable level of precision (in hours, story points, etc.)
- Has an understanding of task prioritization and decomposition
- Clearly understands the difference between severity and priority
Middle
- Can read and understand Test Strategy and Test Plan
- Can actualize or contribute to an existing Test Plan
- Provides estimation for tasks of medium complexity level
- Able to identify major risks and report them to a person responsible for risk management
- Able to make decomposition and estimation of complicated tasks
Senior
- Autonomously prioritizes and estimates tasks of high complexity
- Able to do high-level estimation for projects and knows different approaches
- Can create test documentation (Test Plan, Test Strategy) using best practices
- Able to identify and describe risks, and provide a mitigation plan
Expert
- Knows and uses different estimation approaches (expert, work breakdown, weighted, 3-point)
- Can independently create from scratch Test Plan, Test Cases, Test Suite, CheckList, Test Report, etc.
- Has experience in planning and estimating a complete set of testing activities on 3+ projects
- Can prioritize testing tasks for the project and convince the customer
- Can identify quality risks and provide a plan to mitigate them, and has real-life risk-based testing experience on 2+ projects
- Knows what ROI is and has calculated it 2+ times
- Knows formal models of process improvement and is able to apply them (like TMMi, TPI, and ASPICE)
- Created at least 5+ test plans/test strategies, agreed with the customer, and used them in testing
- Able to explain and sell the benefits of the selected testing approach to stakeholders
02Test Design_
Trainee
Junior
Middle
Senior
Expert
Trainee
- Knows the structure of Test Cases and their difference from Checklists
- Knows the main Test Design Techniques (Equivalence Partitioning (EP), Boundary Value (BV), Decision Table (DT), State-Transition (ST)) - knows 1 or 2
- Can create Test Cases and Checklists with help and directions from senior colleagues on positive and some negative scenarios
-
Knows the differences between:
- Test levels: Unit Testing, Integration Testing, System Testing, Release Testing, Acceptance Testing
- Test types: Functional and Non-functional
Junior
- Understands the most commonly used test design techniques (EP, BV, DT, ST) and uses common test-design approaches when choosing test data
- Can do test design of readable test cases on positive and negative scenarios
-
Knows how to investigate the issue and can provide examples of:
- Test levels: Unit Testing, Integration Testing, System Testing, Release Testing, Acceptance Testing
- Test types: Functional and Non-functional
Middle
- Able to identify and balance test coverage
- Understands the concept of test levels and test types
- Able to identify which Test Cases are more reasonable to put at each test level
- Can do test design on positive and all negative scenarios
- Knows the IEEE standard for test design (29119 p4)
- Uses the most common test design techniques (EP, BV, DT, ST)
- Can define test data
- Able to approach non-functional aspects of the product by suggesting NFT approaches
Senior
- Can review test cases/checklists of colleagues and mentor/consult them, has relevant experience
- Able to perform test design based on given requirements, or according to common sense and experience if exact requirements are not provided
- Optimizes the amount of test data required for Test Case execution
- Involves a risk-based approach in test design
- As a plus, has a preferred approach to test analysis
Expert
- Identifies which test design technique could be used in a specific case, based on experience and risks
- Supervises colleagues on different test design techniques and optimal approaches
- As a plus, has their own approach to test analysis and test design which is proven in 3+ projects
03Test Reporting_
Trainee
Junior
Middle
Senior
Expert
Trainee
- Has basic theoretical knowledge of test reporting.
- Can report their own activities for the day, week, and sprint (tests created, tests executed, bugs created, other activities).
Junior
- Understands 1st level metrics (number): # of defects, # of TCs passed/failed, # of defects by their criticality, etc.
- Can explain what Test Coverage, Test Effort, Defect Distribution, Test Execution, Requirements, and Requirement Coverage are, etc.
- Able to supply a report based on a given template
Middle
- Able to provide a clear structural status for testing activities under their own responsibility.
- Able to read a report and understand the current product quality.
- Understands 1st and 2nd level metrics (relative): pass/fail rate per module/feature, defect leakage rate (e.g., found on PROD/total found * 100%), etc.
- Creates different diagrams to illustrate the quality of testing using metrics.
Senior
- Provides a clear high and low-level structural overview of the testing status for the project.
- Able to differentiate the details of reports depending on the level of the target audience.
- Reports product quality.
- Provides certain, precise, narrative, and easy-to-understand information in reports.
- Regularly collects metrics in daily work, analyzes existing ones for improvements, and implements them for practical usage.
- Understands 1st, 2nd, and 3rd level metrics (distribution): defect leakage dynamics over iterations, dependency between the number of defects and the number of changes over iterations, etc.
- Collects information for metrics from different sources.
Expert
- Able to define reporting standards and templates applicable for different deliveries, and supervise others on the subject.
- Has their own approach and style to report/present the quality, proven with good feedback in 3+ projects.
- Highlights the key things and gives a quick understanding of the essence of the information.
- Assures that reports highlight only valuable information.
- Can apply 1st, 2nd, and 3rd level metrics to the current project, and understands well how to collect and use them.
- Identifies gaps in the reporting process and provides approaches based on test metrics for improving the QA process and quality of the tested application/product.
- Has practical experience of metrics alignment with stakeholders, and implementation in 3+ projects.
04SDLC_
Trainee
Junior
Middle
Senior
Expert
Trainee
- Has basic theoretical knowledge of SDLC and the concept of environments
Junior
- Has theoretical knowledge of Traditional and Agile SDLCs, and practical experience (with at least one)
- Knows the stages of the SDLC and can describe the activities for each stage with examples from practice
- Understands what the build and deployment process is
Middle
- Can describe the advantages and disadvantages of different SDLC models
- Able to describe each stage of a project-specific SDLC in detail
- Knows which environments are used for what
- Understands principles of environment and configuration management
- Knows common test activities and artifacts for each SDLC stage
- Can set up build/deploy/test environments and test tools
Senior
- Has strong knowledge of various SDLCs (knows the pros and cons of each, benefits, challenges), and has wide practical experience
- Has knowledge about the environment lifecycle, from creation to shutdown
- Able to use CI/CD tools, configure them, and set up new flows
- Manages test infrastructure, holds responsibility for the full testing lifecycle
- Identifies gaps in the process, and proposes new approaches to increase speed/quality or reduce resource utilization
- Can introduce methodology practices
Expert
- Participates in SDLC selection for the project, can revise processes and suggest effective ideas for improvements, and works in continuous improvement mode
- Able to adjust the QA process to the current specific SDLC, including project-specific options
- Can set up the QA process starting from static testing and ending with test closure activities
- As a plus, experience and skills are enough to set up agile/scrum project processes and ceremonies from scratch
05Issue Investigation_
Trainee
Junior
Middle
Senior
Expert
Trainee
- Has theoretical knowledge of the structure of a bug and its life cycle
- Able to create a bug report (fill in all required fields), read it, and reproduce the issue in the tracking system
- Knows the difference between verification and validation
Junior
- Able to create a structured bug report with all required information
- Understands the difference between severity and priority
- Can perform defects localization
- Uses available tools to investigate the issue (logs, DBs, console, and DevTools)
- Understands software structure and is able to assume root cause reasons
Middle
- Able to identify reasonable priority/severity
- Able to create a structured bug report (additionally provides bug meta-information: links, tags, builds, modules, children, parents, and details like logs, screenshots, used test data, etc.)
- Can investigate the issue and localize a bug without supervision
- Knows tools to assist with root cause analysis
Senior
- Able to analyze the impact of priority/severity
- Can localize defects in integrated systems
- Understands software structure and can investigate the root cause
- Cooperates with the development team for analysis, follows up on issue fixing progress
- Can read poorly described bug reports and reproduce the issue
- Performs systematization of defects at the end of an iteration
Expert
- Does deep level investigation for the root cause; defines and mitigates potential weak areas ahead, etc.
- Participates in business issues investigation. Provides full information about the bug location, including code references and related issues from the past
- Able to perform localization in an unknown environment by collecting needed information
- Has a deep understanding of systems logic and defect localization principles
- Performs systematization of defects at the end of an iteration to contribute to lessons learned
06Work with requirements_
Trainee
Junior
Middle
Senior
Expert
Trainee
- Has basic theoretical knowledge of what a requirement is and its characteristics. Able to read and understand project requirements.
Junior
- Has experience in requirements testing.
- Can analyze major aspects of a requirement, such as unambiguity, logic, testability, actuality, potential test cases, and integrations. Can analyze different types of requirements: user stories, UML diagrams, flow charts, SRS, and FRS.
- Knows the main criteria for requirements defined by IEEE and can match requirements to IEEE criteria.
Middle
- Provides requirements and specification review and analysis.
- Establishes traceability between requirements and other project artifacts (bugs, TCs, code commits, release notes, etc.)
- Able to analyze dependencies and risks, provide their own vision on weak points, and propose improvements.
- Able to write less formal requirements (e.g., use cases).
Senior
- Able to work with project-level requirements.
- Able to work with low-quality requirements.
Expert
- Provides high-quality requirements and specification review and analysis.
07Work with data_
Trainee
Junior
Middle
Senior
Expert
Trainee
- Has basic theoretical knowledge of what DBs are. Knows the most common databases.
- Understands the data testing process. Has experience with MS Excel and knows the main commands.
Junior
- Understands different data types and is able to define appropriate test ranges under supervision.
- Understands and can analyze a simple data set.
- Can perform simple select queries on a DB.
- Knows common file types such as txt, xlsx, and csv.
- Able to manipulate data with basic tooling functions for analysis.
Middle
- Able to define clear and optimal data sets and scenarios for data quality testing.
- Can analyze and test data flow, transformation, and loading processes (ETL).
- Familiar with at least one database management system.
- Familiar with Excel pivot tables and charts.
Senior
- Performs the analysis of large, complex data sets in short timeframes using optimal approaches and provides valuable results.
- Able to analyze the data workflow and find bottlenecks.
- Understands the impact of data structure usage to identify the algorithmic complexity of processing and common defects that appear because of it.
- Able to generate information using specific data structures as a target, choosing optimal ways to process and save the data.
Expert
- Able to analyze the data workflow and find bottlenecks and provide their own vision on how to improve it.
- Understands the internal architecture of the database and how its engine works.
- Able to generate information using specific data structures as a target, choosing optimal ways to process and save the data.
08Tech skills_
Trainee
Junior
Middle
Senior
Expert
Trainee
- Optional as a plus: knows basic information about Webservices, Web technologies, client-server architecture (how back end works with front end and vice versa), OS, UNIX, SQL, API, REST, Web
Junior
- Basic understanding of how software is created and how it works
- Knows how to use project-specific tools in order to execute test activities (any single tech: OS, UNIX, SQL, API, REST, Web)
- MS Office or Google Docs or command line, etc.
- Knows how to work with any developer tools console
- If applicable - SQL: CRUD
Middle
- Experienced-based:
- Able to explain technology, when it can and should be used/tested, how it works (OS, UNIX, SQL, API, REST, Web)
- Knows how to use test-related tools (Task-management, Test Case Management, Defect Management tools)
- Has a knowledge of domain-specific technologies (FE and/or BE), protocols, logging, DBs, build tools, version-control tools
- Understands types of software (client-server, desktop, web, console, etc.)
- Basic understanding of how OS works (programs, services, files, access)
- Able to solve tasks using any Dev tools console
- If applicable - SQL: can create SELECT queries, knows JOINs and UNIONs
Senior
- Proficient with test-related flows, including custom flow creation, permissions assignment. Replication of a real lifecycle on test-related tools (Task-management, Test Case Management, Defect Management tools)
- Has advanced knowledge of BE/FE surroundings (APIs, SPAs, Logging, DBs, OS, SCM, common architectural approaches)
- Has basic programming skills in any language (Basic knowledge of OOP, at least able to read code)
- If applicable - has good knowledge of working with REST Clients like Postman, Fiddler
- If applicable - knowledge of SQL queries (nested queries, SQL functions, etc.)
- If applicable - knows performance and security testing tools
Expert
- Has architectural knowledge, aware of algorithms, tools, conventions developers use
- Has basic knowledge about Dev framework structure, where components are stored, integration approaches
- Proficient with DBs, Logging, network protocols
- Knows and uses different tools to work with technology. Knows alternatives and can compare/evaluate
- Defines testing tools for projects and the level of testing depth