UI.Vision integration
UI.Vision is an RPA framework that allows us to automate operations in web and desktop applications. It is also multiplatform, being able to run on Windows, Linux and Mac. One of its strengths is that it allows us to perform image and text recognition operations through automated visual tests using a user interface.
As a workflow orchestration platform, in KuFlow what we want is to be able to run robots designed in UI.Vision as part of complex workflows that may involve humans and systems. The robots you design will probably implement communications with the KuFlow API in order to retrieve/store information from the workflow tasks where they are executed, and thus be part of a coordinated way in the elaboration of tasks.
KuFlow offers utilities and examples to integrate a robot of this type.
How to integrate it
UI.Vision robots have a limited way to execute code outside their instruction catalog. The main way is to invoke from the robot an executable or script that is available on the system where it is executed. Therefore the option to implement code for UI.Vision that talks directly to the KuFlow API is also limited. However, we will use an external application, our command line utility "kuflowctl", to be invoked by the robot in order to obtain or store the desired information in KuFlow.
With this approach we could implement a workflow that executes a robot using one of our Temporal engine types or an integration using the API/Webhook type. However, in order to provide you with ready-to-run code and taking into account the benefits that Temporal brings us, we recommend you to use this option which will make your integration work much easier.
Temporal activity to execute a UI.Vision robot
In our SDKs you can find the implementation of a Temporal activity that allows the correct execution of a robot and its control from a workflow. For example, in kuflow-sdk-java libraries, you can find the activity UIVisionActivities. But check the rest of our SDKs for the correct implementation.
The following params are configurable:
| Params | Note |
|---|---|
| autoRunHtml | UI.Vision autorun html |
| macro | Name of UI.Vision macro to execute |
| logDirectory | Ouput directory for logs |
| closeBrowser | UI.Vision option to close the browser when macro is completed |
| closeRpa | UI.Vision option to close the RPA when macro is completed |
| executionTimeout | A timeout for the robot execution, must be less than the timeout of the Temporal activity. |
| taskId | Identifier of KuFlowTask. It can be passed through Workflow. We use the internal UI.Vision vairbale "cmd_var1" variable to provide it. |
| ExtraCommandVariable2 | Custom extra command to pass into the robot. It can be passed through Workflow. |
| ExtraCommandVariable3 | Custom extra command to pass into the robot. It can be passed through Workflow. |
Important aspects
From the activity we must be able to detect if the execution of the robot has been successful or not. Since it is a process that runs inside the browser, the possibility offered by UI.Vision to check this aspect is to read the status written in the robot's log. Therefore, to know if the execution of the robot has finished successfully, the supplied activity checks the status of this log to throw an exception or not to the worker and thus proceed to the logic of retries that our Temporal worker could implement.
Timeout
An important aspect is that we cannot wait indefinitely for the robot in order to detect possible execution failures. For this purpose, we specify a maximum timeout to be configured by the user in the activity. This timeout must be lower than the StartToCloseTimeout configured in the Temporal activity.
Cli - kuflowctl
Normally, within your robot instructions, you will invoke our command line client to interact with the KuFlow API. In this case it is necessary to know if the command execution in the CLI was successful or not. For this purpose, the mechanism provided by UI.Vision when invoking external utilities is to read their output code. Therefore, whenever the output code of the utility invocation is non-zero, we should fail to execute the robot. This is achieved by implementing two command states following each command that is invoked in kuflowctl.
Silent
Due to security limitations in running UI.Vision inside a browser such as Google Chrome, it is important that the kuflowctl call does not produce messages on the standard output of the system. Therefore, whenever you run a command in kuflowctl from a robot, do so with the --silent option.
