CSCE 431 Lecture 18

From Notes
Jump to navigation Jump to search

« previous | Thursday, March 27, 2014 | next »


Test Quality Measures

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle S} is system composed of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n} components denoted Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle C_i} Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle d_i} is the number of killed mutants after applying Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle C_i} 's unit test sequence Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle T_i} to Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle C_i} . Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle m_i} is the total number of mutants of component Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle C_i}

Mutation score Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle MS} for Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle C_i} and its unit test sequence Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle T_i} is given by:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle MS(C_i, T_i) = \frac{d_i}{m_i}}

Test quality Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle TQ} is defined as Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle MS} .

System test quality Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle STQ} is a measure of the quality of the entire test suite for the entire system; given by

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle STQ(S) = \frac{\sum_{i=1}^n d_i}{\sum_{i=1}^n m_i}}

Who Tests?

Any significant project should have a separate QA team

  • avoids self-delusion (blinders; the programmer will seldom catch his own bugs)
  • QA requires a skill-set of its own

Developers typically do their own unit tests

Developers and QA typically perform integration testing

QA Team does system testing

Customer and QA team do acceptance test.


Test Report Classification

Severity

  • severity clasess defined in advance
  • applied to every reported failure
  • analyzes each failure to determine whether it reflects a fault, and if so, how damaging

For example:

  • not a fault (feature)
  • cosmetic
  • minor
  • serious
  • blocking


Status

  • Registered, just reported
  • Open and working
  • Re-opened (thought it was fixed, but not)
  • Corrected
  • Integrated
  • Delivered
  • Closed
  • Irreproducible
  • Canceled


IEEE 829: Testing Documentation

Issue Tracking Systems

Pick one and use it E.g. Traq, Bugzilla


Full-Employment Theorem for Software Testing Researchers

Pesticide Paradox

Every method you use to prevent or find bugs leaves a residue of subtler bugs against which those methods are ineffectual.

Unlike real bugs, software bugs immune to current testing methodologies do not evolve and multiply themselves.

Instead, killing the "easy bugs" allows software to grow with more features and more complexity, creating new possibilities of these subtle defects to manifest themselves as failures.

Therefore, need for new softwere testing technologies is guaranteed ad infinitum Face-smile.svg

Boris Beizer, Software Testing Techniques, 2nd edition, 1990


Code Review

  • Overview
    • Author provides overview of code for reviewers
  • Preparation
    • Reviewers work alone to scrutinize for errors
    • review rate 100–500 lines per hour
    • reviewers can be assigned perspectives (API, user's view, etc)
  • Inspection meeting
    • Reviewer paraphrases code (explains logic)
    • Moderator keeps track
    • Scribe keeps record
  • Post-Meeting discussion


Summary

Having a human read code or use a system detects 80% more faults per (human) hour than testing.

No technique (human or automated) catches more than 75% of bugs; average is around 40%

Retrieved from "https://notes.komputerwiz.net/w/index.php?title=CSCE_431_Lecture_18&oldid=7586"