Test Plan concept


Test Plans define the order in which and the conditions under which Keywords and control structures have to be executed. They also store the directives which will feed keywords input data and ensure that each step produces the expected result.

From a technical standpoint, Test Plans and Composite Keywords are identical. The distinction between them is just a matter of type, which allows to factorize objects and organize a project.

We define a Composite Keyword as a logical grouping of concrete Keywords which are often executed together in a certain order. For example, one can build a composite keyword to represent a complete business case made up of a sequence of concrete Keywords. Then, the Test Plan will reference that Business Case (meaning, the corresponding Composite Keyword) along with other Keywords to simulate a large series of Business Cases, or maybe it will reference that same Composite Keyword multiple times but with different inputs and checks, for example in a ForEach loop, as a ways to "batch test" that Business Case.

We encourage you to make your Keywords as fine granular as possible and to move the logic to your Test Plans as much as possible to reduce the complexity of your project and your maintenance effort.

Test plan structure


A Test Plan is just a tree of objects. These objects can either be Keyword calls (also called Function Calls) or logical controllers, which are used to define when and how the different keywords are supposed to be executed.

When initially creating a Test Plan, you need to pick a logical controller as the root of your tree. By default, you can pick a Sequence for instance, which only ensures the order of its children matches the order they are displayed in the Test Plan Editor.

CreateTestPlan.png

On the right side of the editor, access is given to 3 types of libraries :

  • the control flow library (see tab "Controls") with the different logical controllers.
  • the Keyword library (see the development section) which displays by default the list of all Keyword found on the instance. You can search the Keyword list using the name field.
  • the Other Test plan library, which allow you to call another test plan on the one you are currenlly building.

You can add as many nodes as you want by clicking the + buttons of the controllers and keywords shown in the two libraries. They will then be added to the current node of the Test Plan Tree.

Then, using drag and drop and the upper arrow and lower arrow buttons, you can reorganize the nodes across their parents and place them in the order that you want.

TestPlanUpdate.png

Flow Control library


High-level logic controllers

In addition to the logic you've built inside your script (at keyword level), STEP offers a wide range of logic controllers which can be incorporated to a test plan or composite keyword, thus decoupling the test or business logic from the atomic actions implemented as keywords.

Supported logic controllers

You will find below a description of what each controller does and of its primary properties. Advanced usage will be documented later on. We sometimes refer to them as "control flow structures" or just "controls".

Sequence

Primary use 
simply guarantees the ordering of the child nodes, as displayed in the tree
Primary properties 
none
Example 
simply wrap the child nodes with a Sequence node

For

Primary use 
creates a For loop at execution time and iterates through the child nodes
Primary properties 
item (key of the data accessor), parallel (true/false), threads (number of threads if //), start (initial value), end (final value), inc (increment value)
Example 
item=datapool; parallel=true; threads=10; start=1; end=5; inc=1

ForEach

Primary use 
creates a ForEach loop based on a collection and iterates through the child nodes
Primary properties 
item (key of the data accessor), parallel (true/false), threads (number of threads if //), table (Excel File or SQL Query), folder (JDBC URL)
Example 1 
item=datapool; parallel=true; threads=10; table=D:\ExcelDatapool.xlsx
Example 2 
item=datapool; parallel=true; threads=10; table=select FRUIT_NAME, FRUIT_ID from MYFRUITS; folder=jdbc:mysql://localhost/SID,USER,PASSWORD,com.mysql.jdbc.Driver

(also tested with Oracle ojdbc6.jar) IMPORTANT: You need to copy your driver library (jar file) in the lib folder of the agent to use JDBC functionality.

Session

Primary use 
guarantees that the same driver session will be used for every child node's execution (very important for stateful keywords which rely on a certain driver state as an initial condition)
Primary properties 
selectionCriteria (optional, for affinity patterns)
Example 
leave empty

ThreadGroup

Primary use 
execute the child nodes in a Thread execution pool, allowing concurrent executions of the child nodes
Primary properties 
number of threads, number of iterations per thread, pacing (time to pause between thread next executions)
Example 
number of threads=2, number of iterations per thread=10, pacing=50ms

If

Primary use 
only executes the child nodes if the condition is met
Primary properties 
condition (the expression evaluated to know if the content of the node is executed)
Example 
condition=my_variable === true

Return

Primary use 
delivers a value back to the parent node
Primary properties 
value
Example 
value=3

Check

Primary use 
asserts the input expression, useful for validating the output of the parent node
Primary properties 
expression (the expression to be asserted)
Example 
expression=output.getString("output1")==="val1"

Case

Primary use 
same as in any programing language, works in combination with switch
Primary properties 
value (the value to be potentially matched)
Example 
value=4

Switch

Primary use 
same as in any programing language, works in combination with case
Primary properties 
expression
Example 
expression=my_switch_variable

Set

Primary use 
sets a value to a variable, which can then be accessed throughout the Test Plan and sub Test Plans. Combined with the power of groovy, anything can be achieved in a Set node.
Primary properties 
key (the name of the variable where STEP will write to), expression (the groovy expression to be interpreted, it can be anything)
Example 1 
key=my_connection; expression=groovy.sql.Sql.newInstance('jdbc:mysql://localhost/SID', 'USER', 'PASSWORD', 'com.mysql.jdbc.Driver')
Example 2 
key=my_result; expression=my_connection.rows('select fruit_name, fruit_id from MYFRUITS where fruit_id = 1')

​​You can then pass the value of my_result as a keyword input in the "argument" box: { "input1" : "my_result.fruit_name.first()" } The double brackets will evaluate the content as a groovy expression even though the expression is located inside a json string.

CallKeyword

Primary use 
makes a call to a given function, generally a keyword
Primary properties 
function (json with a key "name" to call an annotated method through reflexion), argument (arguments to be fed through @Arg annotations), token (if a specific token is expected to be used for that call), resultMap (a handle on the resulting data)
Example 
<not_yet_documented>

RetryIfFails

Primary use 
retry mechanism with grace period
Primary properties 
maxRetries (number of retries if step fails), gracePeriod (duration to wait in ms after a failed attempt)
Example 
maxRetries=2; gracePeriod=1000

TestSet

Primary use 
used to group up TestCase's as a single unit and executing them in parallel
Primary properties 
none
Example 
simply wrap your TestCase's with a TestSet controller, parallelisme is derived from the global variable "tec.execution.threads"

TestCase

Primary use 
specific typing of a Test Plan, will activate the top-level panel in the execution view to high-level test case execution monitoring
Primary properties 
none
Example 
simple wrap your Test Plans with a test case

Scheduling Test Plan executions

STEP offers you the possibility to schedule exuections of Test Plans based on JAVA CRON expressions. In order to schedule your Test Plan executions, click on the "Execute" button under the plan tab then on the "Schedule" button.

ScheduleTestPlan.png

You are then invited to enter the CRON expression which will define when the plan will be executed :

CronExpression.png

You can then switch to the "Schedule" tab in order to view, edit and control the scheduling entries you set up :

ScheduleTab.png

Here are some basics examples of CRON expressions. If you are not familiar with, we encourage you to take a look at the CRON official documentation.

ExpressionMeans
0 0 12 * * ?Fire at 12:00 PM (noon) every day
0 15 10 * * ?Fire at 10:15 AM every day
0 * 14 * * ?Fire every minute starting at 2:00 PM and ending at 2:59 PM, every day
0 0/5 14 * * ?Fire every 5 minutes starting at 2:00 PM and ending at 2:55 PM, every day
Tags:
Created by Exense Admin on 2018/01/30 14:26
     
Copyright © exense GmbH
v1.0