Chapter Eight. Working with queues. In this chapter, we'll discuss why work queues are necessary. We'll also talk about queue items traversal and also the other operational parameters. Why work queues? A work queue within blue prism is a configurable list of RPA jobs maintained internally within the application.

It provides the capability to share the workload across multiple machines. A work queue can be shared by multiple processes and similarly, a process can act upon various work queues. As batch progresses, more cases can be provided to the queue and upon completion can be marked as complete or exception based on its generated output. A work queue is managed and maintained from the control room under this queue management option that we have there. We can also view metrics associated with the process, such as volumes of data, system performance metrics, details of an exception, etc, etc. Thus, this feature effectively provides the functionality to share, manage, and report operational work.

Queue items traversal. This is a sequence of steps that must be followed to operate upon work queues. We'll be dealing with them one by one, and finally create a process flow, consolidating all the steps. So let's go to the blue prism application and see how it works. So the first step is to create a queue. We allow our data to a queue and process it based on the sequence of computation that operations are bought performs.

To assign this data, the queue must exist within the blue prism application first. And to create this work queue, we simply go to this system. The manager thing settings thing, and under this menu itself, and we choose work use, right. And I already have a sample queue. But you might have queue one, you might not have any queues. But you can create new queues by clicking New.

And let's name it as a new underscore. Right. And we can provide any name to the word queue. And alongside we have other config options like the max number of attempts that we want the queue, you know, to process the items and key field, which is like in here, and we are going to discuss this in just a few minutes. But this is how our queue is created. Right?

We simply click on apply, and the queue is there. Okay, we head back to the studio and the next step Data assignment. Once the queue has been created, we can now assign data to it by a process flow. Right? So, suppose that we have a collection called marks. Let's create it.

And let's say it has fields called name of type text. Right? And another end of class again text and marks. Data Type as number, right? And let's give some initial values to it as well. So let's put the name as Kevin grass x and marks let's say a 78.

All right, let's add two more rows and Have Smith add to john 95. Okay, so we have the collections data as well in front of us. And what we want to do is now we have a queue that we created, we have to assign this data to the queue. And we can assign collections data itself to the queue. That's how we usually do. So we add an action stage, right here.

Right. And let's say we provide it as add to collection. Right or add collection, not to probably just making it more comprehensible. And in the business object, we choose internal business object work, you These are like the internal business objects which are supported within the blue prism application itself. If in case you cannot find it again, go and import that file, and we should be all set. But these usually do come handy.

Okay, once you have this, you see, there's this option called add to queue, which is like pretty much the first action itself. And we have a few input parameters. And that's like queue name data. And other parameters like differential priority, some tags that we can assign in the status. So what we're gonna do is, we're gonna provide the cue name. So if you remember, let's see what the cue name was, I might have to check exactly was my new cue or new cue something.

And the data let's provide the data first, which is the collections marks collection, right? That's what we created just now. So we got the whole collections data Right, we didn't import any field, we imported the whole collection. And let's just see what the queue name was in the work queues. It's called New underscore queue. Okay.

So we get back to the action stage and we provide the value as okay. So, in here we have provided the data we have provided the queue name, and we have offered input parameters like priority that can be assigned to the items. And differential is like if you want your processing of the data items to to defer to a particular amount of time because this is a date time data type, right? So let's say you want your data items to process like five minutes from now. Right? So then you can set this parameter and until those five minutes are reached our ended the queue would appear as if it Empty or this collection won't appear for sure.

Right, the data belong to this collection will appear in the queue. And after five minutes, that data would get popped into the queue. And then you can operate upon it by retrieving it from the queue itself. Additionally, this stage provides a collection of item IDs as the output parameter. This item ID serves as a unique key to retrieve row data corresponding to the collection assigned to the work queue. This can be understood more by looking at the queue under control room.

But first, let's run this. So we're just gonna run it step by step. Right? And even if we link it, that's all right. That's not gonna make much of a difference. And we've run it and I'm not gonna do anything yet.

What we'll do is Once we go to the control room, we have this, the new q has been added, which we just got from the system, this setting control thing, the system manager here. And under new Q, we see that it's running, it's going to be in the running state on refreshing it, we see that three items have been added. Right? So this data, this item data is accessed by the item IDs that we were talking about, which as a collection can be retrieved as the output parameter can be retrieved through this add to collection add collection. Stage, right? And we're also going to talk about different statuses that an item can exist.

Right now it's in pending state, right? So pending state means like unworked items are marked as pending. They haven't been processed or haven't been operated upon yet. Next is the logged state. So you'll see like a padlock sign once it's in the locked state. And it means that the item is currently being worked upon by the process flow, right.

And then we have the completed state where we'll have a green checkmark. And it's going to denote that the item has been processed successfully. And the last status the item can have is called the exception state, which is going to be a purple flag. And by exception, that means that some abrupt behavior or something, not as per our expectation occurred, and the item has been marked as exception. And what we can do is, let's say if you're relying on the bot right for, for, for a lot of iterations and a lot of work a lot of data that you're dealing with, then you may want to see which all data was computed successfully, which was not. So Martin that has an exception and Looking at just the exceptional the data which has been marked as exception, we can analyze and probably work upon further by either re executing the data or doing it manually I mean depends.

So, those are the different stages that are supported within the work queues are what you items. Step three is going to be work queue item retrieval. So, each work queue item has an item ID right and which is usually is where we what we get from the collection stage and it is something which is maintained internally within the blue prism. Now, when we added to the queue, regarded as a collection but it is optional, since we'll be using our next action, which is gonna be the get next item. So, what we're going to do is the process Low, it's gonna look into the queue. No matter how many data items are there, it's going to look into the queue.

And it's gonna see, it's gonna take or retrieve any data, which is currently in the pending state, like in the three dots sign, right. So right now, all the three data items are in the pending state. So it will take any one of those and will, will process it or we will process it based on whatever actions we are going to add later on. So what we do is we choose the same internal work use as the business object. And we choose get next item. And in this one, we simply provide the cue name.

These are offered by amateurs key filter, if you want to add some filters that you know, let's say you want to work on some specific data items first, or only on those specific items right now within a queue. Then you can provide the key filter and similarly any tags that were assigned when you were adding the data to the collection. They Now you can filter based on that tag as well. Right? So we're going to provide the cue, name here. And we're simply going to get the item ID as just for one because it's going to be item is going to be just one which will be retrieved from the work queue.

So we'll get one item ID, let's create it right here. And we can, we can get the data, like the complete row data, which we assigned just corresponding to the data, the item ID that we just retrieved. We can get all that data in this collection, status and attempts if you want, again, optional, if not, that's alright. On an all should be good. And we just hit OK. So this is going to be the data that will be retrieved.

This is gonna be the item ID that will be retrieved and let's name this also get Next item. Hitting OK. And new underscore Q is the queue name. So everything has been provided correctly, right. And now let's link it as well, in here, we could also set it as the next stage. So one by one, we can, you know, we've discussed this before, we can continue executing. And I haven't executed it yet.

But once we do that, we are going to get an item ID and the data corresponding to that item ID. Let's do that. So you see, some alphanumeric character came up. And you cannot decipher it through naked eye, right? It's something maintained internally within the blue prism application. And you can also see the data.

It's going to be just one row because that's what has been retrieved right now. And you've got the data for Kevin right now. So this has been Retrieved from the queue, it is not from the marks collection. So you can imagine, right? There might be multiple bots working, which are adding data to the queue. And then probably other bots are the same bots processing data from the queue.

So that way you can manage your workload, right? You can have multiple bots working on the same data sets, right? And it's like one central repository where the data is maintained, which needs to be worked upon. So that's the that's the benefit realization of using work queues right. Now, once we have, like, successfully computed like, we can, you know, I know if you want to add, let's say, five marks to each student, right? Then what we can do is, we can simply use a calculation stage.

And in here, we are going to add because this time, it's going to be Data dot marks plus five. And let's probably either plan if you want to store it somewhere else, you can store it. How about we stored in the collection or back in the collection or something? Right, that won't make sense because we don't know the row that will be retrieved from the marsh collection, is it going to be the same or not? So what you can do is probably create a data item or collection data item and store it there. Right.

And you can probably, once you have the collection of that data item, you can simply write that data into the Excel worksheet or depends on whatever business requirement you are trying to solve. So just for the demo purposes, I'm going to put it in a data item so that we can see that the values data being generated are generated correctly. Number doesn't matter data one okay. And so, what happens is once we have successfully computed or successfully performed the operation, which is a very simple operation just adding marks right. So, if you know let's say there is usually once you have worked on a data item within a queue, you would want to see if that went successfully or not and you want to put a decision box to see if the computation was successful or not based on some flag variable or or any conditions that you may put so Once if it was successful, you're going to mark it as complete.

So to mark data item is complete, you choose the internal work object, the business object here, and let's see it here. And marketers come fleet and to market is complete, all we need is just the item ID, no other output nothing else, just the item ID which we got in here, right, it will be marked as complete. And you can also name it as Mark complete. Similarly, if you want to mark something as exception, you use the corresponding activity, the corresponding action from the object over there and the business object in there you should be all set. So, let's set this as next stage. Let's run it one more time.

You see 78 plus five got 83. So the data was computed successfully and stored in the variable and To market is complete, let's actually keep the queue also open. I didn't even show this before, once we refresh the padlock sign, which means currently This item is locked. And once we mark it as complete and we run this let's refresh and it's smog test tech, right. So, what we did was we made it made sure that you know, our process was successful in this case and that you know, we were able to retrieve first data item. Now, it sort of ended right.

So, what you would want to do is, you would want to have a decision stage to check if the item or the if there are any pending items available within the work queue right. So, you can put this like his work the queue empty, right, that means the item ID that was retrieved equals No. If it is yes, right, you simply would want to end the program. If the work is empty, there's nothing else to process, right? So like this. And let's put this like here.

And if no, we can probably go to this stage. And if no, you would want to go back and go back to the next item, right. So let's set an anchor stage here in here, this one and get next item, right, let's run it again now. So queue is not empty yet. So it went it took the next item and when it took the next item The item ID got changed. And if you refresh it, you'll see the second item from the work, you got the padlock sign, which means it is in the lock state.

Now, now you perform the same operation, you're gonna add five marks, you're gonna see the data, one value changes, it got 87, that means the last one was ad, right, whoever this person is, and you're gonna mark it as complete. And if you want to see it got tick mark, and we can complete the rest of the operations in just a single go get next item, adding five marks, you can see the data one it went 200 that means the person was 95 is working empty now went to the next item, which is blank. So there's nothing there's nothing to do. So what we should do is if you see it, because data has because it was empty. So there was nothing to compute upon. Right.

So that's about Very good example, what you should do is, and this is what we do in programming languages as well. We put this stage after the condition. As soon as we get the item, we want to check whether the queue is empty or not. If yes, then only do these computations right? And then if not, then simply end the program. hope that made sense.

So let's make these changes. And I'm going to add a few more things to make this a little better. Let's refresh this whole thing. And we can also see that all of them are marked as complete. So we add the data to the collection, we get the next item. Let's check whether the queue is empty or not.

Okay, if the queue is empty, then simply empty program that is still the same case. If no, let's say we are going to perform some computations, anything could have been there. Right? Okay. And once we perform the computations, we are gonna check whether they were successful or not. I'm just gonna, because these computations could have been anything, so we don't know what it is yet.

But we also would want to check and that's how usually a program is done. What is the success parameter success criteria? So you're going to check whether it was successful or not. Right, put the criteria here in here based on the expression and if it was successful, if it was yes, then mark it as complete right So, something like this and perform the computations as well. And if no, you're gonna mark it as exception, which is gonna be done work queues marked exception here. Here also we provide the item ID.

And we can provide an exception reason we can provide the retry input parameter and keep log which we are going to discuss in a few minutes. And we can once you mark it as exception, we can have a new item ID as well we just generated as an output depending on this retry flight. So we're going to discuss that and let's set the anchor stage in here and If it is not successful marketers exception and Okay, so that's how usually the program is created, we need to check as soon as we get the next item we need to check whether the item is available or not if not simply and the program or if not end then probably whatever the competition is, needs to be performed when the queue is empty. Right those needs to be those competitions need to be added in here those stages and if there are few items available which meet your criteria based on the decision box in here, then simply just computed and mark them as complete or exception This makes sure that you know none other box gets to access the item ID again, right once it has been marked as complete or innocent as an exception, it won't be done.

It won't be computed again, right. And this way we maintain the data integrity between multiple boards as well. Right. So, based on this, I'm also going to add another exercise in this chapter, where I'm going to discuss it a little bit in this video as well, that you have to add data corresponding to any 10. hypothetical employees have a form into a work queue, any work you that you can create in your blue prism, any name you're, you're free to use, right, and the data to be added includes their name, their employee ID, their rank and their salary. And if their salary is more than, let's say 120 thousand dollars, then decrease their salary by 5% and mark the item is complete. However, if the salary is less than 120 thousand, simply mark them as an exception and move on.

It's a very fairly easy example I've given you the criteria as well and the overall structure of this The other solution of this exercise is going to be somewhat what it's what is shown in your screen right now. So, please do try this and also, I want to mention that for software programmers and analogy between this, this whole concept can be drawn with cue data structures. The first in first out data principle, right were on topping, which means retrieving a data item out of the queue. It is always checked if the data structure is empty or not, right. And that's how we do in whether it's Java or c++ or C or mean dotnet anything when you're talking about the data structures, that's how they are done and you can draw the the similarities between the two. Next, we're going to talk about other operational parameters.

The first one is the demo data items. So briefly discussed before. They are are ways to postpone processing of a data item by setting the default until parameter value in the Add to queue action within the work queue objects. So a quick example would be, let's use an expression builder and suppose you want to postpone your processing of data items allotted to a work queue by five minutes from the current date time. Right then in that case, you simply use the date functions which are provided in here and first step would be to get the current date time, which you can get by the now function, simply paste it and then you're going to add five minutes. So to add five minutes, you have like make timespan, which is a function which gets you the timespan from the given days.

So if you provide the parameter values for this function, you don't want it to postpone it by days. You don't want to push opponent by hours. If you want, you can of course, you can provide any value between zero to 23. by five minutes, and zero seconds, just paste it. And here it is, the function is created for us, let's validate the expression the expression is perfectly valid. Now we hit OK. And, and like all the data items will be allotted, but when you're retrieving the data using the get next item, within the process flow, the work you would seem empty. So until five minutes, they won't be visible within the work queue.

Okay, so, that's sometimes used, like when you have a nice, that's a lower priority. You know, items, data items that you may want to work upon later are just to manage the workload. You know, you want to first process the items which are already in the queue which might be a lot by the bots. Then you can provide a differential date and date time and you can postpone the processing of those items accordingly. Okay, next is maximum attempts. So if you remember while we were creating the work queue, we had the option to select the maximum attempts, right.

And this value is one by default but can be increased based on the business requirement. And if the max of 10 values change the data item that hasn't been computed successfully in a work queue is marked as an exception right. But a clone of the same item is inserted into the queue. And this clone item is retrieved by the process using the get next item action and attempted to be computed again. This cloning of data item processes is repeated until the value of the max number of attempts provided in this parameter is met. Right.

So I think As you can see, once you will be going through this item again and again, if the current item that has been taken into consideration hasn't reached the maximum number of attempts, then it will be cloned and put into the work queue. And it will retrieve either by the same bot or by the other bot depending on whichever is working on. But if it was a clone item, all the bots would know that this is like the clone item. So it's not like that. The other bot would probably create another clone of the same item despite the fact that the maximum number of attempts has been reached, right the all the bots will be synchronized and the clones will be created based on the number of maximum attempts provided within the queue because this functionality is at the queue level and not at the bot level or at the process flow level.

Right. Okay, so next parameter we're going to talk about is the retry parameter. So read Try is the input parameter provided within this mock exception action within the work queue objects, right? It works in conjunction with the value of Max attempts option set during the time of the queue creation, which we just discussed right here. If the value of this input parameters set to true, and the maximum attempt limit hasn't been reached yet, the item within the work you will be processed again. However, if it is false, and we have any remaining attempts left as per the max attempts value provided within the work queue, then it overrides the maximum attempt value and marks the item as exception.

Next, we're going to talk about the key block parameter. So this parameter works alongside the retry input parameter. And when true, the new clone item stays in a log status, which provides the opportunity to keep working on this new item. However, if it is false, then the clone item is available the same way as any other pending items within the same work queue and can be retrieved by any bot, which is working on that queue. All right. Next, we're going to talk about the pausing and resuming the work queues for this right here, okay, so we can pause and subsequently release a work queue to manually intervene the retrieval process of items within it.

When paused, the work queue items become temporarily unavailable to the process, and can be reverted back as available only when the work queue is resumed. Right. It's fairly simple. Next, we're going to talk about the queue filters. So similar to excel spreadsheet, we can apply filters to view the relevant queue items within the queue. contents panel right here, let's say any specific item keys there that you may want to, you know, work upon or something else.

Let's do this. Let's see how this works out. We're gonna add data to the collection. And just to refresh it, here it is, right. And now if you wanna, if you have given some priority, you can select them based on the priority like anything greater than one, or anything less than equal to five is needs to be shown here. And if there's any number of attempts that you want to see, or based on the resources that we have, like different computers that are working on the bot, we can see which items are processed by which bots and so on.

And similarly, we have all the other filters that you can check out. All right. The last parameter that we're going to talk about is reports. So for per user and efficient management of a system user, you items can be exported as a basic report. This can be accomplished by simply right clicking here. And we choose the export current view as report.

Right. And once you do that you can export the report like based on whatever data is there on the screen. And using the latest data from the Excel you can export the current pages only on support all the pages using latest data from the database like anything that is provided in here in the queue items. And you can provide the destination in browse wherever you want it and simply finish it and you'll get the report. Okay, so to summarize, the basic fundamental for work queues is that it acts like a central repository for data which can be shared across multiple bots. The capabilities it offers, provide an efficient way to manage workload by sharing data across several working environments, and communicating among each other by marking items as complete or exception and so on.

Okay