RPA integration with UI.Vision - Temporal.io based Worker
This tutorial in a nutshell
In this example, we will build a Temporal-based worker that will allow us to run a robot built using UI.Vision.
What is UI.Vision?
It is an RPA framework that allows us to automate operations in web and desktop applications. It is also multiplatform, being able to run on Windows, Linux and Mac. One of its strengths is that it allows us to perform image and text recognition operations through automated visual tests using a user interface.
But in this example, what interests us is how to integrate it with KuFlow and take advantage of the benefits offered by our platform such as the interaction of this robot with task flows performed by humans. In order to keep the example as simple as possible, we will leave to the user the choice of creating a more complicated workflow with interaction of different tasks and users, and we will focus on running a workflow without human interaction and with a single task whose responsible will be our robot. If you wish, in other examples and documentation on this website, you can see how to orchestrate more complicated flows.
In essence, the robot will take a screenshot of our desktop and upload it to a KuFlow task.
We will choose Java for the implementation of the worker, but remember that this implementation can be done in other programming languages.
Prerequisites
As always, you just need to create a FREE user account in KuFlow (app.kuflow.com) if you don't already have one. Likewise, if you are not yet familiar with our APP, our youtube channel has many videos that will help you in the task.
Create the credentials for the Worker
We will configure an APPLICATION
that will provide us with the necessary credentials so that different external systems can interface with
KuFlow, in this example, the necessary credentials for our Worker.
Go to the Settings > Applications
menu and click on Add application
. We establish the name we want and save. Next, you will get the
first data needed to configure our Worker.
- Identifier: Unique identifier of the application.
- Later in this tutorial, we will configure it in the
kuflow.api.client-id
property of our example.
- Later in this tutorial, we will configure it in the
- Token: Password for the application.
-
Later in this tutorial, we will configure it in the
kuflow.api.client-secret
property of our example.
-
Create the process definition
We need to create the definition of the process that will execute our Workflow. In this section we will configure the KuFlow tasks of which
it is made up as well as the information necessary to complete said tasks, the process access rules (i.e. RBAC), as well as another series
of information. To do this we go to the Setting > Processes
menu and create a new process. You can take a quick look at this
video to get an idea of the user interface for process definition.
A Process Definition with the following data:
- Description
- Free text description about the Workflow.
- Workflow
- Workflow Engine
- KuFlow Engine, because we are designing a Temporal-based Worker.
- Workflow Application
- The application to which our Worker will connect to
- Task queue
- The name of the Temporal queue where the KuFlow tasks will be set. You can choose any name, later you will set this same name in the
appropriate configuration in your Worker. For example:
UIVisionQueue
.
- The name of the Temporal queue where the KuFlow tasks will be set. You can choose any name, later you will set this same name in the
appropriate configuration in your Worker. For example:
- Name
- It must match the name of the Java interface of the Workflow. In our example,
UIVisionSampleWorkflow
is the name you should type in this input.
- It must match the name of the Java interface of the Workflow. In our example,
- Workflow Engine
- Permissions
- At least one user or group of users with the role of
INITIATOR
in order to instantiate the process through the application.- Optional: In order to view administrative actions in the GUI, it may interesting to set the
Manager
role as well for some user.
- Optional: In order to view administrative actions in the GUI, it may interesting to set the
- At least one user or group of users with the role of
A Task Definition in the process with the following data:
- Task "My robot results"
- Description: Free text description about the Task.
- Code: ROBOT_RESULTS
- Candidates:
- This is an automated task, i.e. a system (in this case, our robot) will be in charge of performing it. Therefore, the application we have configured is indicated here as a candidate.
- Element to fill the email:
- Name: Field's label, for example "Screenshot"
- Code: SCREENSHOT
- Type: Field
- Properties: Mandatory, checked.
- Field Type: Document
Source code for our Worker
The source code for this example can be obtained from our java examples repository
However, an alternative methodology could be to download the example template code for the process we are going to implement and compare it with the complete example in our repository. To download the initial template you must access the workflow definition in the App, and once configured, use the download template option.
Main technologies used in the example
This code will serve as a starting point for implementing our Loan Application worker. The requirements for its use are the following:
- Java JDK
- Maven
- To build the example. It is distributed in an integrated way so it is not necessary to install it in isolation.
- Spring Boot and Spring Cloud
- To wire all our integrations.
- Note: There is a version of this example that does not use spring available in our repository.
- IDE
- An IDE with good Java support is necessary to work comfortably. You can use VSCode, IntelliJ Idea, Eclipse or any other with corresponding Java plugins.
To make things simpler, the following dependencies have been used in our example:
-
KuFlow Java SDK
- Provide some activities and utilities to work with KuFlow.
-
Temporal Java SDK
- To perform GRPC communications with the KuFlow temporal service.
-
KuFlow CLI (kuflowctl)
- A command line tool that allows you to interact with the KuFlow Rest API.
-
UI.Vision
-
RPA framework that allows us to automate operations in web and desktop applications
-
Implementation details
The workflow
The entry point to the Workflow execution is determined by the @WorkflowMethod
annotation, in our code the main method would be the
following:
UIVisionSampleWorkflowImpl.java
@Override
public WorkflowResponse runWorkflow(WorkflowRequest workflowRequest) {
this.createTaskRobotResults(workflowRequest);
LOGGER.info("UiVision process finished. {}", workflowRequest.getProcessId());
return this.completeWorkflow(workflowRequest);
}
The structure is very simple:
- Initialize a identifier generator in order to use deterministic Id in idempotent calls.
- Create a KuFlow Task to execute the bot.
- Report completed Workflow.
Here the most relevant thing is the method that the robot executes. The orchestration is simple:
- Create task
- Claim the task
- Execute the robot
- Complete the task
private void createTaskRobotResults(WorkflowRequest workflowRequest) {
UUID processItemId = KuFlowWorkflow.generateUUIDv7();
// Create task in KuFlow
ProcessItemTaskCreateParams createTaskRequest = new ProcessItemTaskCreateParams();
createTaskRequest.setTaskDefinitionCode(TASK_ROBOT_RESULTS);
ProcessItemCreateRequest createRequest = new ProcessItemCreateRequest();
createRequest.setId(processItemId);
createRequest.setType(ProcessItemType.TASK);
createRequest.setProcessId(workflowRequest.getProcessId());
createRequest.setTask(createTaskRequest);
this.kuFlowActivities.createProcessItem(createRequest);
// Claim task by the worker because is a valid candidate.
// We could also claim it by specifying the "owner" in the above creation call.
// We use the same application for the worker and for the robot.
ProcessItemTaskClaimRequest claimRequest = new ProcessItemTaskClaimRequest();
claimRequest.setProcessItemId(processItemId);
this.kuFlowActivities.claimProcessItemTask(claimRequest);
// Executes the Temporal activity to run the robot.
ExecuteUIVisionMacroRequest executeUIVisionMacroRequest = new ExecuteUIVisionMacroRequest();
executeUIVisionMacroRequest.setProcessItemId(processItemId);
this.uiVisionActivities.executeUIVisionMacro(executeUIVisionMacroRequest);
// Complete the task.
ProcessItemTaskCompleteRequest completeRequest = new ProcessItemTaskCompleteRequest();
completeRequest.setProcessItemId(processItemId);
this.kuFlowActivities.completeProcessItemTask(completeRequest);
}
But, wait a minute, where is the implementation of the Temporal activity that the UI.Vision robot executes?