Thursday, April 17, 2008
Graph Matrix
A graph matrix is a square matrix whose size is equivalent to the number of nodes in the flow graph. Each row and column correspond to a particular node and the matrix corresponds to the connections (edges) between nodes. By adding a link weight to each matrix entry, more information about the control flow can be captured. In its simplest form, the link weight is 1 if an edge exists and 0 if it does not. But other types of link weights can be represented:1. the probability that an edge will be executed,2. the processing time expended during link traversal,3. the memory required during link traversal, or 4. the resources required during link traversal.Graph theory algorithms can be applied to these graph matrices to help in the analysis necessary to produce the basis set.
Syntax Testing
In syntax testing, the test-cases, i.e. the input to the software, are created based on the specifications of languages understood by the interfaces of a software. Interfaces have many formats: command-line prompts, files, environment variables, pipes, sockets, etc. An interface has a language which defines what legal input to the interface is and what is not. This language may be hidden or open. The existence of a hidden language is not explicitly realised by developers of the software, but a piece of software might nevertheless read some input data, parse it, and act accordingly. In a broader sense, hidden languages exist also in data structures used to transfer information from a software module to another. An open language is appropriately specified in the software documentation.
The motivation for syntax testing springs from the fact that each interface has a language, whether it is hidden or open, from which effective tests can be created with a relatively small effort. Syntax testing is more likely to find faults from the portions of software responsible for hidden language handling, because open languages must have been explicitly considered by the programmer and thus the input handling portion is likely to be better.
Automated syntax testing requires a formal description of the input language in machine readable format. If the language is hidden, the tester must create a specification for it. Common choices are BNF (Backus-Naur form) and regular expressions. Both are notations to define context-free grammar languages. A sentence is a sequence of bytes which are arranged according to the rules of the language. Context-free languages have traditionally been used in compilers of various sorts to create parsers for input sentences. In syntax testing, a context-free language is the base used to generate sentences. These sentences are then fed, or injected, into the software being tested to see if it accepts them, rejects them or fails to process them altogether.
The selection of test-cases in syntax testing could start with single-error sentences. This is likely to reveal most faults assuming the faults are mutually independent and a fault is triggered by one error in a sentence. After all the sentences with one error are tried, the testing proceeds to pairs of errors, three error combinations, and so on. The number of test-cases grows exponentially by the number of combined errors. Several different kinds of errors can be produced in syntax testing:
Syntax errors
Syntax errors violate the grammar of the underlying language. They are created by removing an element, adding an extra element and providing the elements in wrong order. Syntax errors can exist on different levels in the grammar hierarchy: top-level, intermediate-level and field-level.
Delimiter errors
Delimiters mark the separation of fields in a sentence. In ASCII-coded languages the fields are normally characters and letters, and delimiter are white space characters (space, tab, line-feed, etc.), or other delimiters characters (commas, semicolons, etc.) or their combinations. Delimiters can be omitted, multiplied or replaced by other unusual characters. Paired delimiters, such as braces, can be left unbalanced.
Field-value errors
A field-value error is an illegal field in a sentence. Normally, a field value has a range or many disjoint ranges of allowable values. Field errors can include values which are one-below, one-above and totally out-of-range. Values exactly at the range boundary should also be checked.
Context-dependent errors
A context-dependent error violates some property of a sentence which cannot, in practice, be described by context-free grammar.
State dependency error
Not all sentences are acceptable in every possible state of a software component. A state dependency error is generated by inputting a correct sentence during an incorrect state.
Automatic generation of sentences leads to automatic test design where test-cases are designed by a computer. It is suitable, for example, for stress testing, where software is fed with a large amount of input data.
The motivation for syntax testing springs from the fact that each interface has a language, whether it is hidden or open, from which effective tests can be created with a relatively small effort. Syntax testing is more likely to find faults from the portions of software responsible for hidden language handling, because open languages must have been explicitly considered by the programmer and thus the input handling portion is likely to be better.
Automated syntax testing requires a formal description of the input language in machine readable format. If the language is hidden, the tester must create a specification for it. Common choices are BNF (Backus-Naur form) and regular expressions. Both are notations to define context-free grammar languages. A sentence is a sequence of bytes which are arranged according to the rules of the language. Context-free languages have traditionally been used in compilers of various sorts to create parsers for input sentences. In syntax testing, a context-free language is the base used to generate sentences. These sentences are then fed, or injected, into the software being tested to see if it accepts them, rejects them or fails to process them altogether.
The selection of test-cases in syntax testing could start with single-error sentences. This is likely to reveal most faults assuming the faults are mutually independent and a fault is triggered by one error in a sentence. After all the sentences with one error are tried, the testing proceeds to pairs of errors, three error combinations, and so on. The number of test-cases grows exponentially by the number of combined errors. Several different kinds of errors can be produced in syntax testing:
Syntax errors
Syntax errors violate the grammar of the underlying language. They are created by removing an element, adding an extra element and providing the elements in wrong order. Syntax errors can exist on different levels in the grammar hierarchy: top-level, intermediate-level and field-level.
Delimiter errors
Delimiters mark the separation of fields in a sentence. In ASCII-coded languages the fields are normally characters and letters, and delimiter are white space characters (space, tab, line-feed, etc.), or other delimiters characters (commas, semicolons, etc.) or their combinations. Delimiters can be omitted, multiplied or replaced by other unusual characters. Paired delimiters, such as braces, can be left unbalanced.
Field-value errors
A field-value error is an illegal field in a sentence. Normally, a field value has a range or many disjoint ranges of allowable values. Field errors can include values which are one-below, one-above and totally out-of-range. Values exactly at the range boundary should also be checked.
Context-dependent errors
A context-dependent error violates some property of a sentence which cannot, in practice, be described by context-free grammar.
State dependency error
Not all sentences are acceptable in every possible state of a software component. A state dependency error is generated by inputting a correct sentence during an incorrect state.
Automatic generation of sentences leads to automatic test design where test-cases are designed by a computer. It is suitable, for example, for stress testing, where software is fed with a large amount of input data.
Cause Effect Graph
A cause-effect graph is a directed graph that maps a set of causes to a set of effects. The causes may be thought of as the input to the program, and the effects may be thought of as the output. Usually the graph shows the nodes representing the causes on the left side and the nodes representing the effects on the right side. There may be intermediate nodes in between that combine inputs using logical operators such as AND and OR.
Constraints may be added to the causes and effects. These are represented as edges labelled with the constraint symbol using a dashed line. For causes, valid constraint symbols are E (exclusive?), O (one and only one?), and I (at least one?). For effects, valid constraint symbols are R (Require?) and M (Mask?). (This information needs to be verified.)
The graph's direction is as follows: (XXX: a figure would represent this better)Causes --> intermediate nodes --> Effects
Constraints may be added to the causes and effects. These are represented as edges labelled with the constraint symbol using a dashed line. For causes, valid constraint symbols are E (exclusive?), O (one and only one?), and I (at least one?). For effects, valid constraint symbols are R (Require?) and M (Mask?). (This information needs to be verified.)
The graph's direction is as follows: (XXX: a figure would represent this better)Causes --> intermediate nodes --> Effects
Testing Methodologies - Software testing methodology
Testing methodologies are set of rules or guidelines that are followed to minimize the number of test cases and also provide with maximum test coverage
The following methods are commonly used:
· equivalence partioning
· boundary-value analysis
· error guessing
The following are lesser-used methods:
· cause-effect graphing
· syntax testing
· state transition testing
· graph matrix
Equivalence partitioning
Equivalence partitioning is a systematic process that identifies, on the basis of whatever information is available, a set of interesting classes of input conditions to be tested, where each class is representative of (or covers) a large set of other possible tests. If partitioning is applied to the product under test, the product is going to behave in much the same way for all members of the class.
The aim is to minimize the number of test cases required to cover these input conditions.
There are two distinct steps. The first is to identify the equivalence classes (ECs) and the second is to identify the test cases.
(1) Identifying equivalence classes
For each external input:
(i) If the input specifies a range of valid values, define one valid EC (within the range) and two invalid Ecs (one outside each end of the range).
Example: If the input requires a month in the range of 1-12, define one valid EC for months 1 through 12 and two invalid ECs (month<1>12).
(ii) If the input specifies the number (N) of valid values, define one valid EC and two invalid ECs (none, and more than N).
Example: If the input requires the titles of at least three but no more than eight books, then define one valid EC and two invalid ECs (<3>8 books).
(iii) If the input specifies a set of valid values, define one valid EC (within the set) and one invalid EC (outside the set).
Example: If the input requires one of the names TOM, DICK, or HARRY, then define one valid EC (using one of the valid names) and one invalid EC (using the name JOE).
(iv) If there is reason to believe that the program handles each valid input differently, then define one valid EC per valid input.
(v) If the input specifies a “must be” situation, define one valid EC and one invalid EC.
Example: If the first character of the input must be numeric, then define one valid EC where the first character is a number and one invalid EC where the first character is not a number.
(vi) If there is reason to believe that elements in an EC are not handled in an identical manner by the program, subdivide the EC into smaller ECs.
(2) Identifying test cases
(i) Assign a unique number to each EC.
(ii) Until all valid ECs have been covered by test cases, write a new test case covering as many of the uncovered ECs as possible.
(iii) Until all invalid Ecs have been covered by test cases, write a test case that covers one, and only one, of the uncovered invalid ECs.
(iv) If multiple invalid ECs are tested in the same test case, some of those tests may never be executed because the first test may mask other tests or terminate execution of the test case.
Equivalence partitioning significantly reduces the number of input conditions to be tested by identifying classes of conditions that are equivalent to many other conditions. It does not test combinations of input conditions.
Boundary-value analysis
Boundary-value analysis is a variant and refinement of equivalence partitioning, with two major differences:
First, rather than selecting any element in an equivalence class as being representative, elements are selected such that each edge of the EC is the subject of a test. Boundaries are always a good place to look for defects.
Second, rather than focusing exclusively on input conditions, output conditions are also explored by defining output ECs. What can be output? What are the classes of output? What should I create as an input to force a useful set of classes that represent the outputs that ought to be produced?
The guidelines for boundary-value analysis are:
· If an input specifies a range of valid values, write test cases for the ends of the range and invalid-input test cases for conditions just beyond the ends.
Example: If the input requires a real number in the range 0.0 to 90.0 degrees, then write test cases for 0.0, 90.0, -0.001, and 90.001.
· If an input specifies a number of valid values, write test cases for the minimum and maximum number of values and one beneath and beyond these values.
Example: If the input requires the titles of at least 3, but no more than 8, books, then write test cases for 2, 3, 8, and 9 books.
· Use the above guidelines for each output condition.
Boundary-value analysis is not as simple as it sounds, because boundary conditions may be subtle and difficult to identify. The method does not test combinations of input conditions.
Error guessing
Error guessing is an ad hoc approach, based on intuition and experience, to identify tests that are considered likely to expose errors. The basic idea is to make a list of possible errors or error-prone situations and then develop tests based on the list. What are the most common error-prone situations we have seen before? Defects’ histories are useful. There is a high probability that defects that have been there in the past are the kind that are going to be there in the future.
Some items to try are:
· empty or null lists/strings
· zero instances/occurrences
· blanks or null characters in strings
· negative numbers
One of the studies done by Myers (1979) states that the probability of errors remaining in the program is proportional to the number of errors that have been found so far. This alone provides a rich source of focus for productive error guessing.
The following methods are commonly used:
· equivalence partioning
· boundary-value analysis
· error guessing
The following are lesser-used methods:
· cause-effect graphing
· syntax testing
· state transition testing
· graph matrix
Equivalence partitioning
Equivalence partitioning is a systematic process that identifies, on the basis of whatever information is available, a set of interesting classes of input conditions to be tested, where each class is representative of (or covers) a large set of other possible tests. If partitioning is applied to the product under test, the product is going to behave in much the same way for all members of the class.
The aim is to minimize the number of test cases required to cover these input conditions.
There are two distinct steps. The first is to identify the equivalence classes (ECs) and the second is to identify the test cases.
(1) Identifying equivalence classes
For each external input:
(i) If the input specifies a range of valid values, define one valid EC (within the range) and two invalid Ecs (one outside each end of the range).
Example: If the input requires a month in the range of 1-12, define one valid EC for months 1 through 12 and two invalid ECs (month<1>12).
(ii) If the input specifies the number (N) of valid values, define one valid EC and two invalid ECs (none, and more than N).
Example: If the input requires the titles of at least three but no more than eight books, then define one valid EC and two invalid ECs (<3>8 books).
(iii) If the input specifies a set of valid values, define one valid EC (within the set) and one invalid EC (outside the set).
Example: If the input requires one of the names TOM, DICK, or HARRY, then define one valid EC (using one of the valid names) and one invalid EC (using the name JOE).
(iv) If there is reason to believe that the program handles each valid input differently, then define one valid EC per valid input.
(v) If the input specifies a “must be” situation, define one valid EC and one invalid EC.
Example: If the first character of the input must be numeric, then define one valid EC where the first character is a number and one invalid EC where the first character is not a number.
(vi) If there is reason to believe that elements in an EC are not handled in an identical manner by the program, subdivide the EC into smaller ECs.
(2) Identifying test cases
(i) Assign a unique number to each EC.
(ii) Until all valid ECs have been covered by test cases, write a new test case covering as many of the uncovered ECs as possible.
(iii) Until all invalid Ecs have been covered by test cases, write a test case that covers one, and only one, of the uncovered invalid ECs.
(iv) If multiple invalid ECs are tested in the same test case, some of those tests may never be executed because the first test may mask other tests or terminate execution of the test case.
Equivalence partitioning significantly reduces the number of input conditions to be tested by identifying classes of conditions that are equivalent to many other conditions. It does not test combinations of input conditions.
Boundary-value analysis
Boundary-value analysis is a variant and refinement of equivalence partitioning, with two major differences:
First, rather than selecting any element in an equivalence class as being representative, elements are selected such that each edge of the EC is the subject of a test. Boundaries are always a good place to look for defects.
Second, rather than focusing exclusively on input conditions, output conditions are also explored by defining output ECs. What can be output? What are the classes of output? What should I create as an input to force a useful set of classes that represent the outputs that ought to be produced?
The guidelines for boundary-value analysis are:
· If an input specifies a range of valid values, write test cases for the ends of the range and invalid-input test cases for conditions just beyond the ends.
Example: If the input requires a real number in the range 0.0 to 90.0 degrees, then write test cases for 0.0, 90.0, -0.001, and 90.001.
· If an input specifies a number of valid values, write test cases for the minimum and maximum number of values and one beneath and beyond these values.
Example: If the input requires the titles of at least 3, but no more than 8, books, then write test cases for 2, 3, 8, and 9 books.
· Use the above guidelines for each output condition.
Boundary-value analysis is not as simple as it sounds, because boundary conditions may be subtle and difficult to identify. The method does not test combinations of input conditions.
Error guessing
Error guessing is an ad hoc approach, based on intuition and experience, to identify tests that are considered likely to expose errors. The basic idea is to make a list of possible errors or error-prone situations and then develop tests based on the list. What are the most common error-prone situations we have seen before? Defects’ histories are useful. There is a high probability that defects that have been there in the past are the kind that are going to be there in the future.
Some items to try are:
· empty or null lists/strings
· zero instances/occurrences
· blanks or null characters in strings
· negative numbers
One of the studies done by Myers (1979) states that the probability of errors remaining in the program is proportional to the number of errors that have been found so far. This alone provides a rich source of focus for productive error guessing.
Types of software Testing
There are many types of testing. Generally it is classified as White box testing and Black box testing.
White Box Testing
Testing of a function with knowing internal structure of the program. Also known as glass box, structural, clear box and open box testing. A software testing technique whereby explicit knowledge of the internal workings of the item being tested are used to select the test data. Unlike black box testing, white box testing uses specific knowledge of programming code to examine outputs. The test is accurate only if the tester knows what the program is supposed to do. He or she can then see if the program diverges from its intended goal. White box testing does not account for errors caused by omission, and all visible code must also be readable.
Black Box Testing
Testing of a function without knowing internal structure of the program. Black-box and white-box are test design methods. Black-box test design treats the system as a "black-box", so it doesn't explicitly use knowledge of the internal structure. Black-box test design is usually described as focusing on testing functional requirements. Synonyms for black-box include: behavioral, functional, opaque-box, and closed-box. White-box test design allows one to peek inside the "box", and it focuses specifically on using internal knowledge of the software to guide the selection of test data. Synonyms for white-box include: structural, glass-box and clear-box. While black-box and white-box are terms that are still in popular use, many people prefer the terms "behavioral" and "structural". Behavioral test design is slightly different from black-box test design because the use of internal knowledge isn't strictly forbidden, but it's still discouraged. In practice, it hasn't proven useful to use a single test design method. One has to use a mixture of different methods so that they aren't hindered by the limitations of a particular one. Some call this "gray-box" or "translucent-box" test design, but others wish we'd stop talking about boxes altogether.
Black Box Testing Types
Unit Testing
In computer programming, a unit test is a method of testing the correctness of a particular module of source code. The idea is to write test cases for every non-trivial function or method in the module so that each test case is separate from the others if possible. It is the first test in the development process. The source code is normally divided into modules, which in turn are divided into smaller units called units. These units have specific behavior. The test done on these units of code is called unit test. It is important to realize that unit-testing will not catch every error in the program. By definition, it only tests the functionality of the units themselves. Therefore, it will not catch integration errors, performance problems and any other system-wide issues. In addition, it may not be trivial to anticipate all special cases of input the program unit under study may receive in reality. Unit testing is only effective if it is used in conjunction with other software testing activities.
Integration Testing Integration testing is the phase of software testing in which individual software modules are combined and tested as a group. It follows unit testing and precedes system testing. It takes as its input modules that have been checked out by unit testing, groups them in larger aggregates, applies tests defined in an Integration test plan to those aggregates, and delivers as its output the integrated system ready for system testing. The overall idea, is the "building block" approach in which verified assemblages are added to a verified base which is then used to support the Integration testing of further assemblages. System Testing
System testing is a type of testing where the testing environment is same as that of the production environment. It can be done only when all the integrated modules work fine without bugs. As a rule, System testing takes, as its input, all of the "integrated" software components that have successfully passed Integration testing and also the software system itself integrated with any applicable hardware systems. The purpose of Integration testing is to detect any inconsistencies between the software units that are integrated together called assemblages or between any of the assemblages and hardware. System testing is more of a limiting type of testing, where it seeks to detect both defects within the "inter-assemblages" and also the system as a whole.
Performance Testing Performance testing is a testing that is performed to determine how fast some aspect of a system performs under a particular workload. Performance testing can serve different purposes. It can demonstrate that the system meets performance criteria. It can compare two systems to find which performs better. Or it can measure what parts of the system or workload cause the system to perform badly. In the diagnostic case, software engineers use performance tools such as Mercury Load runner, Empirix e-load to measure what parts of a device or software contribute most to the poor performance or to establish throughput levels (and thresholds) for maintained acceptable response time. In performance testing, it is often crucial (and often difficult to arrange) for the test conditions to be similar to the expected actual use. Performance testing technology employs one or more PCs to act as injectors – each emulating the presence or numbers of users and each running an automated sequence of interactions (recorded as a script, or as a series of scripts to emulate different types of user interaction) with the host whose performance is being tested. Usually, a separate PC acts as a test conductor, coordinating and gathering metrics from each of the injectors and collating performance data for reporting purposes. The usual sequence is to ramp up the load – starting with a small number of virtual users and increasing the number over a period to some maximum. Performance testing is of various types:
1. Load Testing: It is the testing of an application by applying varying loads. The intention is to find the breaking point of the application where it crashes. The load is applied in multiple factors.
2. Stress Testing: It is a form of testing that is used to determine the stability of a given system or entity. It involves testing beyond normal operational capacity, often to a breaking point, in order to observe the results. For example, a web server may be stress tested using scripts, bots, and various denial of service tools to observe the performance of a web site during peak loads. Stress testing a subset of load testing.
3. Soak Testing: It is a kind of testing where an application under test is put under load over a period of time say 48 to 72 hours to check the stability. Generally there can be memory overflow problem if the session connection is not cleaned from the memory.
4. Volume Testing: It is similar to stress testing but generally done for stand alone application where we check for the system by sending huge volume of data across the system. For example if we consider any banking applications, a back up is created for every 5 seconds to avoid crashing of the data. The data from one system is transferred to another system through an intermediate system which monitors the performance of the transfer and also the speed of transfer.
White Box Testing
Testing of a function with knowing internal structure of the program. Also known as glass box, structural, clear box and open box testing. A software testing technique whereby explicit knowledge of the internal workings of the item being tested are used to select the test data. Unlike black box testing, white box testing uses specific knowledge of programming code to examine outputs. The test is accurate only if the tester knows what the program is supposed to do. He or she can then see if the program diverges from its intended goal. White box testing does not account for errors caused by omission, and all visible code must also be readable.
Black Box Testing
Testing of a function without knowing internal structure of the program. Black-box and white-box are test design methods. Black-box test design treats the system as a "black-box", so it doesn't explicitly use knowledge of the internal structure. Black-box test design is usually described as focusing on testing functional requirements. Synonyms for black-box include: behavioral, functional, opaque-box, and closed-box. White-box test design allows one to peek inside the "box", and it focuses specifically on using internal knowledge of the software to guide the selection of test data. Synonyms for white-box include: structural, glass-box and clear-box. While black-box and white-box are terms that are still in popular use, many people prefer the terms "behavioral" and "structural". Behavioral test design is slightly different from black-box test design because the use of internal knowledge isn't strictly forbidden, but it's still discouraged. In practice, it hasn't proven useful to use a single test design method. One has to use a mixture of different methods so that they aren't hindered by the limitations of a particular one. Some call this "gray-box" or "translucent-box" test design, but others wish we'd stop talking about boxes altogether.
Black Box Testing Types
Unit Testing
In computer programming, a unit test is a method of testing the correctness of a particular module of source code. The idea is to write test cases for every non-trivial function or method in the module so that each test case is separate from the others if possible. It is the first test in the development process. The source code is normally divided into modules, which in turn are divided into smaller units called units. These units have specific behavior. The test done on these units of code is called unit test. It is important to realize that unit-testing will not catch every error in the program. By definition, it only tests the functionality of the units themselves. Therefore, it will not catch integration errors, performance problems and any other system-wide issues. In addition, it may not be trivial to anticipate all special cases of input the program unit under study may receive in reality. Unit testing is only effective if it is used in conjunction with other software testing activities.
Integration Testing Integration testing is the phase of software testing in which individual software modules are combined and tested as a group. It follows unit testing and precedes system testing. It takes as its input modules that have been checked out by unit testing, groups them in larger aggregates, applies tests defined in an Integration test plan to those aggregates, and delivers as its output the integrated system ready for system testing. The overall idea, is the "building block" approach in which verified assemblages are added to a verified base which is then used to support the Integration testing of further assemblages. System Testing
System testing is a type of testing where the testing environment is same as that of the production environment. It can be done only when all the integrated modules work fine without bugs. As a rule, System testing takes, as its input, all of the "integrated" software components that have successfully passed Integration testing and also the software system itself integrated with any applicable hardware systems. The purpose of Integration testing is to detect any inconsistencies between the software units that are integrated together called assemblages or between any of the assemblages and hardware. System testing is more of a limiting type of testing, where it seeks to detect both defects within the "inter-assemblages" and also the system as a whole.
Performance Testing Performance testing is a testing that is performed to determine how fast some aspect of a system performs under a particular workload. Performance testing can serve different purposes. It can demonstrate that the system meets performance criteria. It can compare two systems to find which performs better. Or it can measure what parts of the system or workload cause the system to perform badly. In the diagnostic case, software engineers use performance tools such as Mercury Load runner, Empirix e-load to measure what parts of a device or software contribute most to the poor performance or to establish throughput levels (and thresholds) for maintained acceptable response time. In performance testing, it is often crucial (and often difficult to arrange) for the test conditions to be similar to the expected actual use. Performance testing technology employs one or more PCs to act as injectors – each emulating the presence or numbers of users and each running an automated sequence of interactions (recorded as a script, or as a series of scripts to emulate different types of user interaction) with the host whose performance is being tested. Usually, a separate PC acts as a test conductor, coordinating and gathering metrics from each of the injectors and collating performance data for reporting purposes. The usual sequence is to ramp up the load – starting with a small number of virtual users and increasing the number over a period to some maximum. Performance testing is of various types:
1. Load Testing: It is the testing of an application by applying varying loads. The intention is to find the breaking point of the application where it crashes. The load is applied in multiple factors.
2. Stress Testing: It is a form of testing that is used to determine the stability of a given system or entity. It involves testing beyond normal operational capacity, often to a breaking point, in order to observe the results. For example, a web server may be stress tested using scripts, bots, and various denial of service tools to observe the performance of a web site during peak loads. Stress testing a subset of load testing.
3. Soak Testing: It is a kind of testing where an application under test is put under load over a period of time say 48 to 72 hours to check the stability. Generally there can be memory overflow problem if the session connection is not cleaned from the memory.
4. Volume Testing: It is similar to stress testing but generally done for stand alone application where we check for the system by sending huge volume of data across the system. For example if we consider any banking applications, a back up is created for every 5 seconds to avoid crashing of the data. The data from one system is transferred to another system through an intermediate system which monitors the performance of the transfer and also the speed of transfer.
What if there isn't enough time for thorough testing?
Use risk analysis to determine where testing should be focused.Since it's rarely possible to test every possible aspect of an application, every possible combination of events, every dependency, or everything that could go wrong, risk analysis is appropriate to most software development projects. This requires judgement skills, common sense, and experience. (If warranted, formal methods are also available.) Considerations can include:
• Which functionality is most important to the project's intended purpose? • Which functionality is most visible to the user? • Which functionality has the largest safety impact? • Which functionality has the largest financial impact on users? • Which aspects of the application are most important to the customer? • Which aspects of the application can be tested early in the development cycle? • Which parts of the code are most complex, and thus most subject to errors? • Which parts of the application were developed in rush or panic mode? • Which aspects of similar/related previous projects caused problems? • Which aspects of similar/related previous projects had large maintenance expenses? • Which parts of the requirements and design are unclear or poorly thought out? • What do the developers think are the highest-risk aspects of the application? • What kinds of problems would cause the worst publicity? • What kinds of problems would cause the most customer service complaints? • What kinds of tests could easily cover multiple functionalities? • Which tests will have the best high-risk-coverage to time-required ratio?
• Which functionality is most important to the project's intended purpose? • Which functionality is most visible to the user? • Which functionality has the largest safety impact? • Which functionality has the largest financial impact on users? • Which aspects of the application are most important to the customer? • Which aspects of the application can be tested early in the development cycle? • Which parts of the code are most complex, and thus most subject to errors? • Which parts of the application were developed in rush or panic mode? • Which aspects of similar/related previous projects caused problems? • Which aspects of similar/related previous projects had large maintenance expenses? • Which parts of the requirements and design are unclear or poorly thought out? • What do the developers think are the highest-risk aspects of the application? • What kinds of problems would cause the worst publicity? • What kinds of problems would cause the most customer service complaints? • What kinds of tests could easily cover multiple functionalities? • Which tests will have the best high-risk-coverage to time-required ratio?
How can it be known when to stop testing?
This can be difficult to determine. Many modern software applications are so complex, and run in such an interdependent environment, that complete testing can never be done. Common factors in deciding when to stop are: • Deadlines (release deadlines, testing deadlines, etc.) • Test cases completed with certain percentage passed • Test budget depleted • Coverage of code/functionality/requirements reaches a specified point • Bug rate falls below a certain level • Beta or alpha testing period end
What's a 'test case'?
A test case is a document that describes an input, action, or event and an expected response, to determine if a feature of an application is working correctly. A test case should contain particulars such as test case identifier, test case name, objective, test conditions/setup, input data requirements, steps, and expected results.
Note that the process of developing test cases can help find problems in the requirements or design of an application, since it requires completely thinking through the operation of the application. For this reason, it's useful to prepare test cases early in the development cycle if possible
Note that the process of developing test cases can help find problems in the requirements or design of an application, since it requires completely thinking through the operation of the application. For this reason, it's useful to prepare test cases early in the development cycle if possible
What's a 'test plan ?
A software project test plan is a document that describes the objectives, scope, approach, and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the 'why' and 'how' of product validation. It should be thorough enough to be useful but not so thorough that no one outside the test group will read it. The following are some of the items that might be included in a test plan, depending on the particular project: • Title • Identification of software including version/release numbers • Revision history of document including authors, dates, approvals • Table of Contents • Purpose of document, intended audience • Objective of testing effort • Software product overview • Relevant related document list, such as requirements, design documents, other test plans, etc. • Relevant standards or legal requirements • Traceability requirements • Relevant naming conventions and identifier conventions • Overall software project organization and personnel/contact-info/responsibilties • Test organization and personnel/contact-info/responsibilities • Assumptions and dependencies • Project risk analysis • Testing priorities and focus • Scope and limitations of testing • Test outline - a decomposition of the test approach by test type, feature, functionality, process, system, module, etc. as applicable • Outline of data input equivalence classes, boundary value analysis, error classes • Test environment - hardware, operating systems, other required software, data configurations, interfaces to other systems • Test environment validity analysis - differences between the test and production systems and their impact on test validity. • Test environment setup and configuration issues • Software migration processes • Software CM processes • Test data setup requirements • Database setup requirements • Outline of system-logging/error-logging/other capabilities, and tools such as screen capture software, that will be used to help describe and report bugs • Discussion of any specialized software or hardware tools that will be used by testers to help track the cause or source of bugs • Test automation - justification and overview • Test tools to be used, including versions, patches, etc. • Test script/test code maintenance processes and version control • Problem tracking and resolution - tools and processes • Project test metrics to be used • Reporting requirements and testing deliverables • Software entrance and exit criteria • Initial sanity testing period and criteria • Test suspension and restart criteria • Personnel allocation • Personnel pre-training needs • Test site/location • Outside test organizations to be utilized and their purpose, responsibilties, deliverables, contact persons, and coordination issues • Relevant proprietary, classified, security, and licensing issues. • Open issues • Appendix - glossary, acronyms, etc.
Subscribe to:
Posts (Atom)