Software Testing with Microsoft Test Manager 2012 and Lab ManagementBrian Keller

DEV-B327

Who is this session for?• For manual testers who want to improve

their productivity and collaborate more effectively.

• For developers who want to start automating functional UI tests.

• For teams who want to improve software quality by producing better bugs and/or adopt agile testing.

• For teams who want to automate the provisioning of dev/test labs and build-deploy-test workflows.

This session is not for….• …unit testing.• …fakes / mocking.• …web / load testing.

ALM Virtual Machines:http://aka.ms/ALMVMs

http://tinyurl.com/ALM2012 http://tinyurl.com/ProTFS2012

USS Yorktown, SmartShip

• crew member entered 0 in a data entry field, caused a “divide by 0” error that shut down propulsion

• dead in the water for 2hrs 45mins

Ariane 5 Flight 501

• re-used code from Ariane 4, but took a different flight path because of different booster design

• conversation from 64bit float to 16bit signed int caused overflow (exception handler was disabled for perf reasons)

• cost: > $370 million @ 1996 prices

Vasa

• sank after sailing less than 1 nautical mile out of Stockholm

• specifications were faulty, failing to take into account enough ballast to keep the ship upright

• test teams revealed that it was unstable, but launch was rushed, and QA was ignored

Training

Conditioning

Climbing

Training Phase

Cost of Failure

Development

Test

Release

Software Phase

Cost of Bugs

Generalist Specialist

Manual Testing Some scripting

Creates scripts to set up lab, create data

Strong scripting skills

Some coding skills

Strong coding

Develops automated testing procedures

Expert coding skills

Tester Segmentation

Black Box Testing

White Box Testing

API Testing

70% of testing happens heremajority of test tools target here

What usually happens when a testerfinds a bug?

Graphic by Jeff Atwood (CodingHorror.com)

Unified ALM Solution

Team Foundation

Server

Planning SCM Work Item Tracking

Testing and Lab

Management

Build Automation

Reporting and

Analytics

Microsoft Test Manager 2012

demo

(1) Get source (2) Compile projects

(3) Copy build to running environment

(4) Run deployment scripts for each machine

(5) Create environment snapshot

(6) Executeautomated tests

(7) Send testresults

(8) Publish results toTeam Foundation Server

Build – Deploy – Test

Team Build

Team Foundation Server

Test Controller

VMM Agent Test Agent Test Agent

VM Host Web Server (VM)

Database Server (VM)

Virtual Environment

VMM Agent Test Agent Test Agent

VM Host Web Server (VM)

Database Server (VM)

SCVMM ENVIRONMENT

Test Architecture

Test Agent Test Agent

Web Server Database Server

STANDARD ENVIRONMENT

Test Agent Test Agent

VM Template 1

VM Template 2

VM TEMPLATE LIBRARY

Team Foundation Server

System Center Virtual Machine Manager

(SCVMM)

Team Build

Test Controller

SCVMM vs. Standard EnvironmentsSCVMM Environment Standard Environment

Build-Deploy-Test

Manual Testing

Snapshot / Restore

Share Bug Snapshots

Network Fencing

3rd Party Virtualization

Physical Machines

Lab Management

demo

Lord Kelvin, 1824-1907

“…you cannot manage what you cannot measure.”

Team Foundation Service details

Included for certain paid MSDN subscribers:

Free Plan for up to 5 users

Additional information at http://tfs.visualstudio.com

* Capability in preview – limits may apply. Authoring load tests requires Visual Studio Ultimate 2013 Preview.

Version control (TFVC or Git) Comment on changesets & commits Work item tracking and tagging Team rooms Agile planning tools Feedback Management Agile Portfolio Management* Build* Web-based test case management* Load testing*

New!

New!

New!

New!

© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.