Find Out How

SOASTA CloudTest provides the analytic tools you need to examine and understand the results of your tests. Tests that you create can be evaluated and revised based on analysis of the tests themselves. You can use default dashboards, or quickly and easily setup new dashboards and widgets similar to the way you create widgets on portal sites such as Yahoo! and Google.

SOASTA CloudTest provides many charts you can place into one or more dashboards. For example, charts for Response Time, CPU, and Send/Receive Rate per Second. You can also create custom widgets from your own data sources.
SOASTA CloudTest’s analytic features also include the ability to combine server monitoring of its own features, as well as that of related servers and databases, as well as providing the user the ability to refine logging properties to fine tune the analytic information itself.

Status

Status presents test composition status in terms of success or failure. In addition, the number of messages played and errors received and duration of the test including start/end time are presented.    
The first part of the Status message indicates whether the test composition completed or failed to complete. If the test failed to play to the end, return to the Composition Editor to determine why. The second part of this expression indicates whether the test failed on the basis of other criteria, such as validation.
Test Success is green.

Failure is red. Whereas a success message is brief, a failure message may present as much information as possible. Additional information makes a fix or workaround easier to find.

A test that is still playing is indicated by yellow and the missing second part of the expression. The result can always be opened during test play but represents a snapshot in time.

Environment

By environment, we mean the entire set of conditions under which a Test Composition has been run. For any test, this includes the number of SOASTA CloudTest Maestro Servers running (in the case of multi-threaded, multi-server deployments of SOASTA CloudTest), the location of the Maestro Server under which the test was run including the system name, the hardware of the machine where Maestro is installed, including memory, model, clock speed, and processors, and the operating system and version. 

Analytic Charts

SOASTA CloudTest includes many interactive charts to help you make sense of message response time, processing time, and the rate at which messages are sent and responses received. Additional analytic charts are available via the Central > Monitors feature.

Response Time

• Mouse over any item in the Response Time chart to view its details.

• To zoom on a region click-and-drag a rectangle. Double-click to unzoom the view

CPU

In the Central Processing Unit (CPU) chart, system resource use by percent is presented on a timeline.

For CPU results to populate, a Monitor must be created using a defined Monitoring Server Group and then linked to the given test composition using Composition Editor > Test Composition Properties.

Send and Receive Rate

In the Send and Receive Rate chart, blue indicates send rate, while orange indicates receive rate.

• Mouse over any item in the Send and Receive chart to view its details.

• To zoom on a region click-and-drag a rectangle. Double-click to unzoom the view
• Use the Chart Legend to display only sends or only receives. Click the View icon for Send or Receive to change the view. 

Refining Logging Properties

SOASTA CloudTest provides verbose logging of all its activities by default. For functional testing, nearly all of this detail will prove useful. However, for load testing, verbose mode may be a lot more information than is useful for the test. SOASTA CloudTest provides Logging settings that can be customized with the intensive nature of load testing in mind.

1. In order to examine logging settings within the Composition Editor, click Properties in the Navigator (lower left) list, then select Composition (in the Property Type list).

2. Click the Logging tab as shown below.

The default logging settings, all of which are verbose, are displayed in the screen shot to the right.

3. In the Change All drop down, select Error.

• In the Test Composition drop down, set Statistics or Verbose. We recommend Statistics for load tests.
  
  Note:  Leave the Record Statistics for Analytics checkbox checked. Unchecking this box will disable the Analytics Dashboard.