Welcome back. In this video, we are going to understand how to use file module how to configure file connector and how to trigger a flow when a new file is added to a directory. So, I'll tell you my requirement in my C drive I have a folder called as files inside this I have input whenever I keep any file into this folder, I want the flow to be triggered and process the file that is my simple requirement. So actually, in the lab documentation, which is shared to you, I have sales dot CSV l LP open it it is simple CSV file, which contains some sales related line each line is one record. I want to process all this records by pulling for a file in this input directory. So let us see So I'll be working on this project using file module.

I'll drag and drop. There's no file module here, I had to add a module. file, drag it to the left side. So the file module should be added here in the class path. Now there is an operation called as on new or upload for updated file. I'll drag and drop flow with this.

Yeah, so I'll configure the properties of this one. For this also added to configure configuration for every endpoint as I told you, there is one configuration here let's check what tasks in this character configuration. Yep, it is asking for a working directory. The base directory for all this so I want to see called files with a working directory. Working Directory is nothing but a big Directly Okay. Now, here there is one more attribute which is asking directly.

I'll give this as input so I want to pull four files in C colon files slash input. And what are the recursive checkbox? recursive means suppose under c colon files colon input slash input. If there are some recursive directories directory inside our directory answer directory, if I want to go into all his directives recursively and pick the files, then I can simply say recursive by selecting this checkbox. Okay. A lot of other options in this file and point I will discuss next, but I want to drag and drop the logger to display the content of the file.

Okay, I'll drag and drop the logger and I just configure it to log the payload. Simply I'll write hash payload. Okay. Now let me start. Okay, the application got deployed. Now what I'll do is, I'll copy this sales dot CSV and paste it inside the input.

Let's observe the console. See, the file content is printed. I'll clear the console. Again, the file control is printed out their console. It's actually what is happening is the final Still there, the files still there. Again, when the file endpoint polls, the file is that login ticket.

It's actually if you see here in my file endpoint configuration, I have configured it to Paul by default. For a fixed frequency scheduling strategy, you can see it is configured to pull up our scheduled frequency of 5000 milliseconds, I'll make it as five and time notice seconds so I want you to call for every five seconds now. So after every five seconds, what it is doing is polling the input directory and reading it, but the file is not deleted, the file is not deleted in the next poll also same file will be processed. So I want the file to be deleted. Is there an option? Let's see.

Yes, there is In a section called as post processing section, where you can configure auto delete equals true. Okay, now I will save my file. I will restart. Okay, it started now what I'll do is I'll keep my sales are CSV again the input directory. It should get processed and it should be deleted after it is processed. Let's wait.

Yep, see it got deleted. And now you can see the console right? So if I just configure the end Point to delete, it'll automatically delete once a file is processed. Sometimes I don't want the file to be deleted, so I'll make it as false. I want the file to be moved to some other directory. So what I have done is in the sequel files, I have output directory I want the files to be moved to this one.

So what I'll do is here c colon slash files slash output as a mood, I want the files to be read to move directly so I'll go and create a directory called us. mode. Okay. And in this mode directory, what should be the file name? Actually, to retrieve the file name, what I can do is I can say attributes dot file name, so this will give me the original file. Name and I want to concatenate it with say, dot backup.

I want the file to be moved and it should be renamed to a file with extension dot bak. Now let us see I made these auto auto deltas false Now let us see. Better I will restart it and killing it. So let me restart. Okay, it got started so I'll copy the sales dot CSV into files. So it should be automatically picked up and it should be mowed right now after five seconds Three seconds is over what is happening Yeah, it got picked up and in the mood directory let us see yes I can see says our CSV dot backup.

So now you understood the option of how to move the process to file. Okay. I don't want it to be moved. I want the file and also I don't want the file to be deleted from the input folder. So what to do, there is one option in file module called as watermarking. So first of all I will remove this moto directory and renamed to okay.

So here there is something called as watermarking which is disabled. So here I'll configure it as modified timestamp. I like explain you what is this watermarking right now. So again, I will restart after enabling the watermarking. I'm going to restart Oh, it's already running. So I want to stop and start.

So I'll wait for it to start. Okay, it started now. I'll put the file says our CSV in the input. I see that the file got processed once. I'll clear this and I will check if in the input that there is a file still there. Yes.

Still the file is there. But after five seconds, Let us wait it should not be processed. So actually, it will not get reprocessed. Even though the file is present the on new or upload file connector, it understood that the file was already processed, how actually I selected watermarking murders modified timestamp, what is watermarking actual the watermarking concept came from maybe I can say the dams, if you have seen any water dams which are created, which are actually present there will be water level every day the people who are working on at the dams they wanted to record the water level every day. So, every day to port for time, they will they used to create in olden days, they used to create one Mark they used to mark the level of water and then next day they used to compare what is the level of water today.

So, the word watermarking came from that point simply Here in case of fire endpoint, the fire endpoint what it will do is now it will whenever the fire is processed, here it is configured to store. It's configured to store what modified timestamp, the timestamp of the file the modified timestamp of the file. So the file is not modified, it will not be reprocessed. Even after the next poll, it will check what is the modified timestamp. If the file is not modified, it will not reprocess so what I'll do is right now, I just go to the input directory and I'll open the sales and I will delete one line modifying it. So now let us see if it gets reprocessed.

And observe the console. Yes, it got reprocessed is it. So that means after the fight is modified when the file in point went for the next poll it chequered. What is the updated timestamp of this file? Is this greater than whatever I have saved earlier? Yes.

So it will reprocess the file that is what watermark mode. So, it can also configure as created timestamp. So, if you configure the watermark in more as created timestamp, what the file endpoint will store the file creation time, even though your file is modified. Now, it will not be picked up with the next poll, because the Create timestamp will always remain same. so if you want the file to be processed only once, even though the file is not final file is modified you don't want it to process. Then you can select Create a timestamp, or if you want the file to be processed whenever it is modified, it can select modified timestamp Right.

So now you understood about watermarking also. In the next video, we'll see how to write the files. See you in next video.