When we looked at how to combine several commands into pipelines, and pass the output of one command to the other, if we work on data files such as we did in the previous lecture, we may want to process the output and store it in another file. This process is also known as ETL. From extract transform load, we can do it in the terminal using the concept of output redirection. I will print the content of the random sh script into run to.sh script. Essentially, this will make a copy of the file but it will not copy the file permissions though. Now, let's say we want to add an additional line to the file.
We can do it using the double grades assigned to append to the file. Let's now see what's in the second transcript we created. We can execute the file now but first we need to add executable permission. I will now execute the file And then we'll see that it also printed the content of the current directory using LS. Let us now execute the script and redirect the output to a log file. We don't see any output printed to the terminal since it was all written to the output log.
If we would like to execute a script and ignore the output as maybe we don't care about it, we can use this special file slash dev slash now and redirect all the output into it. This file is essentially a virtual device which disregards any information sent to this file. The first character in the file permission for this file is C, which says it is a character device. Its role is mostly to accept any input and ignore it just as we did before. In the terminal like we just now there are two parents streams. One is called STD out for the regular output, and the other one is called STD error for errors.
If we print the output to a file when we drag the first STD out string to the file. Now let me modify the script and add an additional line that from the printer narrow to the STD error stream. I can use a special notation for the STD error stream which denotes the stream with ID two. I can save the file and quit. Let me execute it now first. Looks like I'm in a narrow looking fix it I see what the problem is.
I should make a space before the ampersand sign. I can fix it. I say we can quit. Let me run it again. Now it printed their line. If not redirect the output to a file.
The main output is printed to the file, but the arrow is still printed to the terminal. I cannot fix it by telling the executor that I want to redirect the output of the second stream to the first stream. I can do it by appending the arguments with the following notation to more than ampersand one which says that they want to print the output of the stream with any tool which has STD error to the stream with ID one, which has to be out which in return will be redirected to the output dot log. In our can see that there was printed to the output log. The ampersand sign denotes the idea of the streams. Without it the errors would be redirected to a file name one or two.
This is already a bit advanced. topic, but you might encounter this usage in some scripts. And now we will know what this peculiar combination means. In this in previous lecture, you learn two very powerful concepts of pipelines and output redirect, which is in the pocket of any professional Linux user. Now, analyzing large text files would be really no problem for you.