06 August 2007 - 1.7.5 | home user-guide eclipse jbossws intellij netbeans maven 1.X/2.X PDF files forums bugs sourceforge

Home
Getting Started
User Guide

- Overview
  - Tabbed Desktop
  - License Management
  - Preferences
  - XPath Selection
  - Script Library
- Workspaces
- Projects
- Interfaces
  - Interface Viewer
- Operations
- Working with Requests
  - Form Editor
  - Outline Editor
  - Message Inspectors
  - Attachments/Inline Files
- Functional Testing
  - Getting Started
  - TestSuites
  - TestCases
    - Test Requests
      - Assertions
      - Editor Wizards
    - Property Transfers
    - Conditional Gotos
    - Groovy Scripts
      - Examples
      - GroovyUtils
    - Properties Step
    - Delay Step
    - Mock Response
    - DataSource
    - DataSource Loop
  - Property Expansion
- Load Testing
  - Getting Started
  - Limit & Strategies
  - Execution
  - Assertions
  - Diagrams
  - JMeter comparison
- Mocking
  - Getting Started
  - Mock Services
  - Mock Operations
  - Mock Responses
- Usage Scenarios
  - Data-Driven Testing
  - Template-Driven Testing
  - Interactive Testing
  - Surveillance Testing
- Tool Integrations
  - Code Generation
  - WSDL Generation
  - WS-I Tools
  - Apache Tcp-Mon
- CommandLine Tools
  - TestCaseRunner
  - LoadTestRunner
  - MockServiceRunner
  - ToolRunner
- Keyboard shortcuts

IDE/Tool Plugins
Search soapUI site
Project Documentation

- About soapUI
- Project Info
- Project Reports

## LoadTest Execution

When running a LoadTest soapUI internally creates a complete copy of the underlying TestCase for each thread, allowing each TestCase to maintain its own state and properties. Depending on which limit and strategy has been selected, loadtesting will proceed as configured until the LoadTest terminates due to one of the following:

- It has reached its configured limit
- It has been cancelled by the user with the "Cancel" button on the LoadTest toolbar
- It has been cancelled by a LoadTest Assertion when the maximum number of allowed errors for that assertion has been passed

During execution, the following statistics are periodically collected and displayed in the Statistics Table:

- `min` - the shortest time the step has taken
- `max` - the longest time the step has taken
- `avg` - the average time for the test step
- `last` - the last time for the test step
- `cnt` - the number of times the test step has been executed
- `tps` - the number of transactions per second for the test step, see Calculation of TPS/BPS below.
- `bytes` - the number of bytes processed by the test step
- `bps` - the bytes per second processed by the test step
- `err` - the number of assertion errors for the test step

Collection/calculation of this data is performed asynchronously (ie independently from the actual TestCase executions) in soapUI, so it should not affect the actual loadtesting performance.

The Total row shows the statistics for the TestCase itself.

### Step Time calculation

soapUI internally uses the `System.nanoTime()` method for determining the actual time taken by a test-step. The following table describes how these times are calculated for the available teststep types:

Step Type | Duration
--- | ---
Request Step | The request duration is calculated depending on the settings in the LoadTest Options dialog (as described there).
Groovy Script | The actual script evaluation
Properties Step | The time to load/save the properties (if configured)
Delay Step | The steps delay
Property Transfer | The time it took to transfer all properties
Goto Step | The time it took to evaluate the goto-conditions

### LoadTest Assertions

When running a LoadTest, the configured assertions are applied and will generate an error in the log if not met:

Assertion errors can be double-clicked and will show a view displaying the underlying TestStep state that failed the assertion. For example the following will be shown for Request that failed an assertion:

### Calculation of TPS/BPS

Depending on the setting of the "Calculate TPS.." option in the LoadTest Options dialog, TPS and BPS are calculates as follows:

- NOT based on actual time passed (default):
  - TPS : (1000/avg)*threadcount, for example avg = 100 ms with ten threads will give a TPS of 100
  - BPS : (bytes/cnt) * TPS, ie the average number of bytes per request * TPS. For example a total number of received bytes of 1000000 for 100 requests with a TPS of 100 would give (100000/100 * 100) = 1000000 BPS
- Based on actual time passed:
  - TPS : Time passed in TestCase / CNT / 1000, ie a TestCase that has run for 10 seconds hand handled 100 request will get a TPS of 10
  - BPS : Bytes / Time passed / 1000, ie a TestCase that has run for 10 seconds and handled 100000 bytes will get a BPS of 10000.
  Please note the time-passed value used is the time for the entire testcase, not each step for itself. This can bias the TPS/BPS values quite a lot for those teststeps that take a small amount of the TestCases total time

The option to base on actual time passed can be usefull when setting a testcase delay using the Simple LoadStrategy, which will generally give a lov average, but the actual transactions per second will not be equivalently high (since there is a delay). Selecting this option will calculate TPS using (time-passed/cnt) instead.

Next: LoadTest Assertions