Introduction to Software Testing and Assurance

Cold Test [email]

Go to 3320HTML\ptw1011ex1.htm. This is a web based calculator. Test the calculator.

Constraints (maximum!):

• 5 minutes planning time
• 15 minutes actual test time
• 10 minutes summary report

Do not spend more than 30 minutes on this project!!!  You may vary the time between phases a bit, but keep the time limited to 30 minutes. Include time spent on this assignment (by phases) in your report.

Email me the assignment at tkombol@uncc.edu by noon on Wednesday. Be sure to include the Subject line of 3320 Cold Test . The assignment may be an attached document or included as the email content.

TCS Chapters 1 and 2 - 2 assignments [email]

• Chapter 1
  1. Why should tests and their results be carefully documented and logged?
  2. What are boundary conditions and how are they used in testing?
• Chapter 2
  1. Why can't a non-trivial program be tested exhaustively?
  2. During testing are you proving the program works correctly or are you finding and reporting errors in the program? Justify your answer.

Answer each question with a short paragraph.

Email each assignment separately to tkombol@uncc.edu. and rmukherj@uncc.edu  Be sure to include the Subject line of 3320 TCS Chapter 1 and 3320 TCS Chapter 2 for each email. Do not include the assignment as an attached document. Put the answers as text in the email.

1. What is acceptance testing (from a testing standpoint)?
2. Why is it better to find errors as early as possible in the software development process?
3. What is glass (white) box testing?
4. What is black box testing?
5. Are black box and white box testing complementary, non-related, or mutually exclusive? Justify your answer.

Answer each question with a short paragraph. For each question:

• 1pt - good answer
• 1pt - well written (proper spelling and grammar)

Email me the assignment at tkombol@uncc.edu and rmukherj@uncc.edu.  Be sure to include the Subject line of 3320 TCS Chapter 3. Do not include the assignment as an attached document. Put the answers as text in the email.

TCS Chapter 4 [email]

1. Who should be the center of all quality efforts? (2 pts)
2. Why? (8 pts)

Answer the second question with a short paragraph and email me the assignment at tkombol@uncc.edu and rmukherj@uncc.edu. Be sure to include the Subject line of 3320 TCS Chapter 4. Do not include the assignment as an attached document. Put the answers as text in the email.

HBS Chapter 1 [email]

How to Break Software Chapter 1 Questions 1 and 2 on page 15. 

Scan the Black Box Testing Project.  Consider a potential product to test using the Black Box Test techniques.  Answer questions 1 and 2 with that product in mind.  Limit question 1 to 2 types of users for each of the classes.  This is an individual exercise.  Use judgment on the time spent for each question.  I don't think more than 10 minutes is required for each type and part.  Do create reasonable sized lists.

Organize your answers in an easy to read format, and don't forget to mention which application you are referencing.

Send the answer as an email me at tkombol@uncc.edu and rmukherj@uncc.edu. Be sure to include the Subject line of 3320 HBS Chapter 1 Do not include the assignment as an attached document. Put the answers as text in the email.

TCS Chapters 5 [email]

1. 2 pts.) What is the main purpose of writing Problem Reports?
2. 6 pts.) Name 3 reasons to clearly document how to reproduce a found bug.
3. 2 pts.) On page 83 of TCS, the inset bar says to never use original copies of data. Why?

Answer each question with a short paragraph and email me the assignment at tkombol@uncc.edu and rmukherj@uncc.edu. Be sure to include the Subject line of 3320 TCS Chapter 5. Do not include the assignment as an attached document. Put the answers as text in the email.  

Test Fest 1 report [paper]

Write a report on your testing of the NVu product.  Include at least:

1. Overview of Test Fest 1
   • Brief review of test environment, product tested
   • Include information from product research  (on Web)
2. Describe procedures followed
   • Include partners name
3. Assessment:
   1. Test environment
   2. Product
4. Summary
• Good/bad product
• Areas for further/specialized testing
• etc.

Deliverable: A copy of your report with the above information.  The report should look professional. Part of the grade will be grammar, spelling and style.

The title of this homework is 3320 Test Fest 1  

Design a simple Problem Tracking System (PTS). Imagine this system being used in a test facility for a moderately sized project.

The PTS will have two aspects:

• To record data from errors found by testers and others
• To present appropriate summary reports of errors recorded in the PTS database 

For this exercise the summary reports will be restricted to two views:

• one for errors assigned to a programmer  (e.g. only the responsible programmer will only be able to see the PRs assigned to him)
• one for all open errors to be sent to and review by the Project Manager

The input form may be paper based, e.g. a form (e.g. produced and printed by Word or Excel) to be manually filled in, or based on an electronic form (e.g. an HTML web page or a stand-alone program).

• Record Errors Form
  Either paper or electronic based:
  • Paper based
    Develop a paper form using word processor or the like and print it. Assume copies of this form will be filled in manually by testers and the data later entered into the PTS database by a clerk. The paper form will then be filed. It may be updated later as more information is learned.

    Make sure all the fields needed are defined to gather relevant data. Also, make sure the form is well organized to make it easy to write the data in the proper places in the form and to make it easy for someone to enter this data into another system later. The organization of the form will also indicate required vs. optional information.

    To go with the input form, design two report forms:

    • One for developers to know what PRs are open against them
    • One for Project Managers showing all open problems by severity and age.

    Deliverable: copies of the Problem and Report forms in paper format, handed in on class. Have both clean (unfilled form) and an example of each with sample data.

  • Electronic based
    Read the requirements for the Paper based system.  Based on those requirements, develop an electronic form that could be used to automatically enter data into a database. This form does not need to actually enter data to any database, but should have an "enter" key or button to pretend it did. Use any tool available in the 335 lab to create this form. Suggestions: Excel, Access, Visual Studio, any Web Application etc. Remember, this form will be a prototype, not functional. One of your design decisions will be whether to make one form to gather and report data, or make separate forms.
• Report Form
  • Programmer's list:
    List only data that is of interest to a specific programmer.  This list should contain only PRs assigned to the programmer, with information that will help him decide which PS is the most important for him to be working on next.  Show reports for at least two different programmers.
  • Project Manager list:
    List information that will help the Project Manager understand the status of the project.

All input forms should be no more than 1 page. The "paper" input form should be able to record all information on 1 sheet of paper.  The electronic form should record all information on one equivalent of a web page with minimal scrolling.

The report form should be able to show all information for one PR on no more than one sheet of paper. A tabular form may be used with columns holding the data and each row a separate PR.  Have both clean forms (no data) and a sample of one filled in by pen or pencil with simulated data with 4 or more PRs.

The purpose of this exercise is to start you thinking of how you will capture error information and later be able to review any errors status.  Think of how to most effectively format each of the forms.  These forms will be used to record and report the results of the Glass Box and Black Box projects.

These forms may be reviewed in class.

Deliverables:

• Print copy of an empty input form
• Four filled out examples of the report forms
• Two sample printout of Programmers List for two different programmers
• Sample printout of Program Managers Report

The title of this homework is 3320 TCS Chapter 6

TCS Chapter 7 [prep work for Test Fest 2]

This assignment is a lead in to Test Fest 2.  For Test Fest 2 your team will test in depth a function of NVu.

Test Case (or Test Sets):

Consider the Tables in NVu.

Devise a set of tests for checking the table functions in an HTML document. The test should consider the following:

• Boundary conditions:
  • What are the variables? (Hint: rows and columns are only two of the variables.  There are at least 6 others.)
  • What are their limits?  (maximum and minimum sizes)
• Equivalent classes:
  What equivalence classes are there for tables. (see Figure 7.1 on page 127).
• What paths can be used to insert or create tables?
• Address whether the same tests can be used for the various paths, or whether each path will require a unique path.

Example:

• Test Case: Tables
• Simple Table
• TableTest1
• TableTest2
• ...
• Simple Table
Bad Data
• TableBadTest1
• TableBadTest2
• ...
• Large Table
  • TableLTest1
  • TableLTest2
  • ....
• Embedded Tables
  • TableEmbedTest1
  • TableEmbedTest2
  • ...

Deliverables to be included in your report for Test Fest 2:

• The answers to the above considerations (explicitly).
• Your documented test sets.
• Results of the tests.
  • Simply Passed with date and tester if it worked
  • PTR filled if it failed

Be ready to test NVu with your test set in Test Fest 2.  A copy of your Test Case needs to be handed at the start of Test Fest 2.  It will be graded later with your Test Fest 2 report.  Note: you may upgrade/enhance your test cases for Test Fest 2 if testing finds problem areas.  Make sure this paper has a cover sheet.

TCS Chapter 9 and 10 [email]

1. What is localization?
2. Briefly discuss problems that must be addressed when testing "localized" software.  What are the key issues?
3. Why bother to "test" manuals?

Send the answer as an email me at tkombol@uncc.edu and rmukherj@uncc.edu. Be sure to include the Subject line of 3320 TCS Chapter 9 and 10. Do not include the assignment as an attached document. Put the answers as text in the email.

1. See 02/17/2009 above as basis
2. Execute Test Cases for Chapter 7
   • At least one Test Case for tables
   • Test at boundaries
   • Don't forget tests for error handling (e.g. correctly rejecting bad data/input)

Report:

1. Overview
2. Describe test environment
• Environment
• List Test Case(s)
• Name of test case
• What the test case covers
• etc.
• For each Test Case list it's Tests
• Name of test
• Why this test
• Input(s) used or tested
• Output(s) used or tested
• Explicit instructions to execute test
• Expected result(s)
• Criteria used to judge if test passed (no error detected) or failed (found an error)
3. Document tests
• Results of executing all tests
• Test ID / Name of test
• Who run
• When run
• Pass/fail (PTR Number?)
• For each error detected generate a Problem Tracking Report
• PTR Number
• Product/version tested
• Test ID
• Who ran
• When ran
• Short description
• Details to reproduce
• Comments
• etc.
4. Summary

TCS Chapter 11 - Automation [email]

Automating tasks has plus and minus aspects. Discuss what tradeoffs are made when deciding to automate or not. Specifically:

• What problems does automation solve?
• What potential problems do you need to be made aware of when doing automation?

Send the answer as an email me at tkombol@uncc.edu and rmukherj@uncc.edu. Be sure to include the Subject line of 3320 TCS Chapter 11. Do not include the assignment as an attached document. Put the answers as text in the email.

Chapter 12 - Professionalism in Testing [email]

Why is it important to thoroughly and professionally document the test process.  Include the hazard of over documenting in your discussion.

Send the answer as an email me at tkombol@uncc.edu and rmukherj@uncc.edu. Be sure to include the Subject line of 3320 TCS Chapter 12. Do not include the assignment as an attached document. Put the answers as text in the email.   

Arrive at class with a test plan for a specific aspect of NVu to test.  The test plan must be typewritten and turned in by the end of the class.  You may want to consider 2 copies, one to turn in, one to take notes on as you test.  As you test you may have the opportunity or need to expand your tests.  Have a cover sheet for the part you turn in.

In Class:

Execute your Test Cases for TF 3

Minimum Report:

1. Overview
• Describe which aspect of NVu is being tested
2. Describe test environment
• Environment
  • Partner
  • Windows
  • MacBook Pro
  • Etc.
• List of Test Cases and Tests
  • For each Test:
  • Name of test
  • Why this test
  • Input(s) used or tested
  • Output(s) used or tested
  • Explicit instructions to execute test
  • Expected result(s)
  • Criteria used to judge if test passed (no error detected) or failed (found an error)
3. Document tests
• Results of executing all tests
• Test ID / Name of test
• Who run
• When run
• Pass/fail (PTR Number?)
• For failed tests: generate a Problem Tracking Report
• PTR Number
• Product/version tested
• Test ID
• Who run
• When run
• Short description
• Details to reproduce
• Comments
4. Summary

Keep in mind you will be exercising these same tests the Mac environment for Test Fest 4

The title of this homework is 3320 Test Fest 3  

Chapter 13 - User Interface (UI) Freeze [email]

The UI Freeze is a typical major milestone in the development process.

1. Why is a UI Freeze done?
   Center on the aspects concerning documentation, marketing, and testing.
2. Explain the impact to testing if a change has to be made to the UI after the UI Freeze was done (particularly if GUI has been changed and some testing is automated).

Send the answer as an email me at tkombol@uncc.edu and rmukherj@uncc.edu. Be sure to include the Subject line of 3320 TCS Chapter 13. Do not include the assignment as an attached document. Put the answers as text in the email.

• Retest Test Fest 3 in the Mac OS X environment
• Same minimum report requirements as Test Fest 3 but add/change
• Environment
• In summary, also note the "sameness" or differences in what you found during testing
• any significant look and feel differences
• any testing results different

The title of this homework is 3320 Test Fest 4

HBS Chapter 2, 3 and 4 [email]

From HBS Chapter 2:

How to Break Software Chapter 2 Questions 1 and 3 on page 55. 

Use the program you've selected to do the black box testing to answer this question.  Limit the exercise to 2 features.  I would suggest making a table for each feature and listing the inputs for each.  For each feature, also list what output (data, state change, etc.) is associated with each input.

Note that question 3 builds on question 1.  You can submit the homework as one answer.  Also, HW in chapter 3 and 4 will build on this table.  I suggest making a Word document to hold the data in a table.  You can then cut and paste the table into the email you send me or send it as an attachment.

Organize your answer in an easy to read format, and don't forget to mention which application you are referencing.

add the following from HBS Chapter 3:

How to Break Software Chapter 3 Questions 2 and 4 on page 79.

This adds onto Chapter 2 Question 1 above.  For the two features you used above add the extra columns for data structures and computations done for each feature and its inputs.  Use N/A where appropriate.

Again, organize your answer in an easy to read format, and don't forget to mention which application you are referencing.

and finally add the following from HBS Chapter

HBS Chapter 4 Question 1 page 104.

Use the same application as used for the HBS Chapters 2 and 3 parts.  Include a quick overview of a description of the application (e.g. name and what it does).

Send the answer as an email me at tkombol@uncc.edu and rmukherj@uncc.edu. Be sure to include the Subject line of 3320 HBS Chapter 234 Attachments are allowed for this assignment. 

Estimating Errors [email]

Show your formulas and calculations for full credit!

Also for each of the estimating techniques below, state what assumptions must be true of the errors for the estimates to be accurate.

1) Bebugging:

Your test group has inserted 250 bugs into an application under test.  While testing, the testers have found 56 of the bebugged errors.  They have also found 226 other errors.  Estimate:

1. total number of remaining "real" or orignal errors
2. total number of remaining errors in the product under test.

2) Lincoln Index:

Assume two test teams working independently on the same project.

Team one: finds 1242 errors.
Team two finds 944 errors.

Estimate the total errors if :

1. there are 97 common errors found
2. there are 627 common errors found

3) Capture for removal:

Your test team finds 937 errors in its first round of testing.  All those errors were fixed and assume no new errors introduced.

On the next round of testing estimate the total number of original errors if

1. 74 errors are found
2. 742 errors are found

Send the answer as an email me at tkombol@uncc.edu and rmukherj@uncc.edu. Be sure to include the Subject line of 3320 Estimating.  Do not include the assignment as an attached document. Put the answers as text in the email.

Chapter 15 - Quality Assurance [email]

From page 346 in TCS: "Quality Assurance groups 'assure quality.'  You can't do that by testing"

Why can't quality be tested into a product?

This assignment will be graded mainly on grammar and persuasiveness.  Make sure your argument is clear and well written.  One of two forms is suggested for the assignment:

• Essay format covering the subject
• Inter-office memo to your supervisor explaining why concentrating on testing only does not assure quality

Other formats may be acceptable, but they must be well written and compelling.

Appropriate references are a plus.

Send the answer as an email me at tkombol@uncc.edu and rmukherj@uncc.edu. Be sure to include the Subject line of 3320 TCS Chapter 15. Do not include the assignment as an attached document. Put the answers as text in the email.