26 September 2007 - 1.7.6 | home user-guide eclipse jbossws intellij netbeans maven 1.X/2.X PDF files forums bugs sourceforge

Home
Getting Started
User Guide

- Overview
  - Tabbed Desktop
  - License Management
  - Preferences
  - XPath Selection
  - Script Library
- Workspaces
- Projects
- Interfaces
  - Interface Viewer
- Operations
- Working with Requests
  - Form Editor
  - Overview View
  - Outline Editor
  - Message Inspectors
  - Attachments/Inline Files
- Functional Testing
  - Getting Started
  - TestSuites
  - TestCases
    - Test Requests
      - Assertions
      - Editor Wizards
    - Property Transfers
    - Conditional Gotos
    - Groovy Scripts
      - Examples
      - GroovyUtils
    - Properties Step
    - Delay Step
    - Mock Response
    - DataSource
    - DataSource Loop
  - Property Expansion
- Load Testing
  - Getting Started
  - Limit & Strategies
  - Execution
  - Assertions
  - Diagrams
  - JMeter comparison
- Mocking
  - Getting Started
  - Mock Services
  - Mock Operations
  - Mock Responses
- Usage Scenarios
  - Data-Driven Testing
  - Template-Driven Testing
  - Interactive Testing
  - Surveillance Testing
- Tool Integrations
  - Code Generation
  - WSDL Generation
  - WS-I Tools
  - Apache Tcp-Mon
- CommandLine Tools
  - TestCaseRunner
  - LoadTestRunner
  - MockServiceRunner
  - ToolRunner
- Keyboard shortcuts

IDE/Tool Plugins
Search soapUI site
Project Documentation

- About soapUI
- Project Info
- Project Reports

## SoapUITestCaseRunner

The runner is defined in the `com.eviware.soapui.tools.SoapUITestCaseRunner` class and takes the path to the soapUI project file containing the tests and a number of options:

switch | description
--- | ---
e | The endpoint to use when invoking test-requests, overrides the endpoint set in the project file
h | The host:port to use when invoking test-requests, overrides only the host part of the endpoint set in the project file
s | The TestSuite to run, used to narrow down the tests to run
c | The TestCase to run, used to narrow down the tests to run
u | The username to use in any authentications, overrides any username set for any testrequests
p | The password to use in any authentications, overrides any password set for any testrequests
w | Sets the WSS password type, either 'Text' or 'Digest'
d | The domain to use in any authentications, overrides any domain set for any testrequests
r | Turns on printing of a small summary report (see below)
f | Specifies the root folder to which test results should be exported (see below)
j | Turns on exporting of junit-compatible reports, see below
a | Turns on exporting of all test results, not only errors
o | Opens the generated report in a browser (soapUI Pro only)
i | Enables soapUI UI-related components.

The distribution contains a testrunner.bat script for running tests in the bin directory, for example;

    testrunner.bat -hlocalhost:8080 -a -fresults c:\projects\my-soapui-project.xml

will run all the tests defined in the my-soapui-project.xml file against the specified host and export all test results to the "results" folder (will be created if not available).

### Launch Dialog

For getting started with the command-line testrunner, soapUI includes a "Launch LoadTestRunner" action available from Project/TestSuite/TestCase popup menus:

The options available include those described above and also add options to set if the project shoud be saved, the path to the runner (leave empty for same as installation), etc. When launching the specified testrunner will be launched from the command-line as configured and the output can be followed:

The output contains the actual command-line command used for launching the runner, which can easily be copy/pasted for external reuse

### JUnit Integration

It is fairly easy to invoke the testrunner from your own junit-tests;

    public void testRunner() throws Exception
    {
    	SoapUITestCaseRunner runner = new SoapUITestCaseRunner();
    	runner.setProjectFile( "src/dist/sample-soapui-project.xml" );
    	runner.run();
    }

The `runner.run()` call will throw an exception if an error occurs. If you want more control over your integration / error-reporting, a specific TestCase could be run as follows:

    public void testTestCaseRunner() throws Exception
    {
    	WsdlProject project = new WsdlProject( "src/dist/sample-soapui-project.xml" );
    	TestSuite testSuite = project.getTestSuiteByName( "Test Suite" );
    	TestCase testCase = testSuite.getTestCaseByName( "Test Conversions" );

    	// create empty properties and run synchronously
    	TestRunner runner = testCase.run( new PropertiesMap(), false );
    	assertEquals( Status.FINISHED, runner.getStatus() );
    }

### Reporting and Exporting

The SoapUITestCaseRunner has basic reporting functionalities, including the possibility to create junit-compatible xml-reports using the -j switch. Also, it will during execution print diagnostic information and if the -r switch was specified print a small summary:

    testrunner.bat jbossws-soapui-project.xml -stest -ctesting -r -a -fmyresults

produced the following output:

    SoapUI 1.5beta2 TestCase Runner
    12:33:10,042 INFO  [SoapUITestCaseRunner] setting projectFile to [jbossws-soapui-project.xml]
    12:33:10,042 INFO  [SoapUITestCaseRunner] setting testSuite to [test]
    12:33:10,042 INFO  [SoapUITestCaseRunner] setting testCase to [testing]
    12:33:10,583 INFO  [WsdlProject] Loaded project from [jbossws-soapui-project.xml]
    12:33:11,915 INFO  [SoapUITestCaseRunner] Running soapui tests in project [jbossws]
    12:33:11,915 INFO  [SoapUITestCaseRunner] Running soapui suite [test], runType = SEQUENTIAL
    12:33:11,925 INFO  [SoapUITestCaseRunner] Running soapui testcase [testing]
    12:33:11,935 INFO  [SoapUITestCaseRunner] runing step [Groovy Script - init boss]
    12:33:12,335 INFO  [SoapUITestCaseRunner] runing step [Properties]
    12:33:12,335 INFO  [SoapUITestCaseRunner] runing step [Transfer Values - set boss]
    12:33:12,716 ERROR [SoapUITestCaseRunner] Transfer Values - set boss failed, exporting to
     [myresults\test\testing\Transfer-FAILED.txt]
    12:33:12,716 INFO  [SoapUITestCaseRunner] runing step [request step]
    Retrieving document at 'http://lpt-olma:8080/ws4ee-samples-server-ejb/Organization?wsdl'.
    12:33:13,407 INFO  [SchemaUtils] Loading schema types from [http://lpt-olma:8080/ws4ee-samples-server-ejb
     /Organization?wsdl]
    12:33:13,407 INFO  [SchemaUtils] Getting schema http://lpt-olma:8080/ws4ee-samples-server-ejb/Organization?wsdl
    12:33:13,787 INFO  [SoapUITestCaseRunner] Assertion [Schema Compliance] has status VALID
    12:33:13,807 INFO  [SoapUITestCaseRunner] Finished running soapui testcase [testing], time taken = 1882ms
    12:33:13,807 INFO  [SoapUITestCaseRunner] Skipping testcase [testcase2], filter is [testing]
    12:33:13,807 INFO  [SoapUITestCaseRunner] Skipping testcase [Copy of testing], filter is [testing]
    12:33:13,807 INFO  [SoapUITestCaseRunner] Skipping testcase [Copy of Copy of testing], filter is [testing]
    12:33:13,807 INFO  [SoapUITestCaseRunner] soapui suite [test] finished in 1892ms

    SoapUI 1.5beta2 TestCaseRunner Summary
    -----------------------------
    Time Taken: 1895ms
    Total TestSuites: 1
    Total TestCases: 1
    Total TestSteps: 4
    Total Request Assertions: 1
    Total Failed Assertions: 0
    Total Exported Results: 4

By default, the testrunner exports only failed results to a text file, the `-a` option will export all results instead. For example the file for a Request TestStep will be as follows (slightly modified to fit):

    Status: OK
    Time Taken: 55
    Size: 448
    Timestamp: Sun Mar 12 12:45:57 CET 2006
    TestStep: request step
    ----------------------------------------------------
    Encoding: UTF-8
    Endpoint: http://lpt-olma:8080/ws4ee-samples-server-ejb/Organization
    Username: asd
    Password: dfsdfdsf
    Domain: asdasd
    ---------------- Request ---------------------------
    <soapenv:Envelope xmlns:sam="http://org.jboss.test.webservice/samples"
    	xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
       <soapenv:Body>
          <sam:getContactInfo>
             <String_1>testsd1141581163341</String_1>
          </sam:getContactInfo>
       </soapenv:Body>
    </soapenv:Envelope>
    ---------------- Response --------------------------
    <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
    	xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
       <soapenv:Body>
          <ns1:getContactInfoResponse xmlns:ns1="http://org.jboss.test.webservice/samples">
             <result>The 'testsd1141581163341' boss is currently out of office, please call again.</result>
          </ns1:getContactInfoResponse>
       </soapenv:Body>
    </soapenv:Envelope>

Exported files are written to a file named <TestSuite>-<TestCase>-<TestStep Name>-<Count>-<Status>.txt in the current or specified folder. An example filename is "TestSuite 1-TestCase 1-Request Step 1-0-OK.txt" (the count is added since a TestStep can be invoked several times within the run of a TestCase). As of soapUI 1.7.5, response attachments are also exported, named <TestSuite>-<TestCase>-<TestStep Name>-<Count>-attachment-<AttachmentCount>.<Extension>, where the extension is created from the attachments content-type if possible (otherwise .dat will be used).

### JUnit Reports

Adding the -j switch produces xml-reports in the same format that are exported by the junit ant task with the xml-report format. Currently, soapUI maps TestSuites to report-packages and TestCases to report TestCases. For example running the following:

    testrunner.bat -j -ftestresults sample-soapui-project.xml

produces a "TEST-Test Suite.xml" file in the specified testresults folder. This result can be further transformed using the ant junitreport task:

    <junitreport todir="./testresults">
       <fileset dir="./testresults">
          <include name="TEST-*.xml"/>
       </fileset>
       <report format="frames" todir="./testresults/html"/>
     </junitreport>

Which produces an output as follows:

Errors are shown as follows:

Wrapping everything into one ant-target for win32 would be as follows:

    <target name="soapui-report">

    	<exec dir="C:\\Program Files\\eviware\\soapUI-1.7.5\\bin" executable="cmd.exe">
    		<arg line="/c testrunner.bat -j -fg:\\reports g:\\projects\\sample-soapui-project.xml"/>
    	</exec>

    	<junitreport todir="g:\\reports">
    	  <fileset dir="g:\\reports">
    	    <include name="TEST-*.xml"/>
    	  </fileset>
    	  <report format="frames" todir="g:\\reports\\html"/>
    	</junitreport>

    </target>

Next: SoapUILoadTestRunner