Lab 33 :- DQS( Data Quality Services ) (SSIS)

MSBI Step by Step Training Lab 33 :- DQS( Data Quality Services ) (SSIS)
43 minutes
Share the link to this page
Copied
  Completed
In this video we will try to understand what is DQS(Data Quality Services).

Transcript

In this video, we will try to understand what is d q s, dq S stands for data quality services. Now, if you remember when I started this training, I said that msbi is all about data data and data. And till now, what we are doing we are using the three important pillars of msbi that is SSIS, SSIS and SSRS to ensure that is this data is converted into information right. So, in other words, we are using SSIS to do ETL we are using SSIS to do analysis and we are using SSRS to do reporting. But now, think about it whatever we are doing till now, this is done programmatically In other words, this all thing is done by programmers that means people who are having high technical knowledge of msbi like you and me, but let us try to think logically at the end of the day, this data generated by end users.

So end users are responsible for good data and for bad data. And more the data is bad, then more our work increases in exercise, because then we have to ensure that this bad data is removed, right? So if we can give some kind of power or some kind of tool, or some kind of a service right to the end user when the end user controls this good data and bad data. In other words, the end user ensures that whatever data comes to us is proper than a lot of our work can be reduced. And that's what exactly take us does. dq S is a service, you know which end user as well as the programmer can utilize to ensure that his quality of the data is good.

Now take this simple example of a product excel sheet. Now this product excel sheet is entered by end users. So the end user is sitting in a shop. Or I'd say that our end users, a lot of users who are entering into this single excel sheet, right. So they are sitting in a shop and they're making entry of the product name and the total sales done. But you can see that you know, there are a lot of problems with this Excel sheet.

You can see your entry of shoes, and then the value is 2000. Close. Now, this is plural, right, close 3000 and then there is cloth singular 3000. Then we have item code 98. Now this item code 98 is nothing but close actually. But this item code 98 is, you know, some kind of internal code, which is used in the shop.

But then there is a smart end user who actually rather than putting the direct product name, he's putting the item code. Now look at this other kind of a user. Rather than entering shoes. He's entering the brand name like Reebok and Adidas and then there are some end users Those who are really tensed, I don't know what but you know, they are entering no data at all. So you can see that whenever data comes right, you know, they come in such kind of a format and most of the time. The problem with the data are synonyms means, for example, shoes, and Reebok means one of the same thing, right?

Close and close means one of the same thing. So, if somebody takes this data and it tries to do analysis, humans end up with the wrong analysis, isn't it? And second, our work in SSIS will increase tremendously. So, because this bad data is generated by the end user, it would be great you know if the end user can rectify this data. So, end user is the best person to rectify this data because at the end of the day, our technical program cannot really recognize that clothes or clothes or same thing. Reebok and shoes are same thing, right.

So, we need The end users to actually go and clean this data. So, before it comes to SSIS right, it would be great that if this data can be cleaned, so, first let us try to understand if we give this Excel sheet to an end user, how do we solve this problem or how will he go about cleaning this Excel sheet. So, if we given this Excel sheet without any tool, he will look at this Excel sheet and he will say you know what, this shoes and Reebok means the same thing. So, he will go and say that this is actually shoes. So, in other words, he will actually go here and start correcting this data in as per his knowledge, so, in other words, the end user has some kind of a knowledge base in a very and he goes and he identifies that close or close is the same thing.

Item code and closes the same thing. So, you will actually go and start manipulating the data, depending on his knowledge base depending on his experience. Depending on his domain knowledge what he has, and this knowledge base has to be improved or I'd say it has to be grown, as the time goes by, because as time goes by more bad data comes in, for example, let's say somebody comes in and he puts this entry, he goes and he puts entry like this. So, he goes to the Excel sheet, and there is an entry which is like this copra right. Now, if you look at this World Cup, this is more of a Hindi word. And if the end user is Italian, let us say, who's collecting the seat?

He has no idea that what this word means, isn't it? So, he will go and he will ask some other end user who is from India saying that what is this he will say that this means close and then he will go and correct it. So next time What will happen is, he knows that this word here means close, right? So in other words, even the end user increases his knowledge base. As time goes by, because bad data keeps on coming in. So, if we are giving this power to the end user, then the first thing we need to give to the end user is a knowledge repository a knowledge base, where in the end user can go and start saying that okay, this means this, this means this, so that he can use that knowledge base time and again, to clean the data as well as improve the ability of the tool to detect bad data.

When the session started, I said that end users are responsible for bad data and for good data. So now, because you're giving this power to the end user, they are the same people. So now, they will be responsible for good knowledge base and bad knowledge base. So we should have some kind of approval mechanism in place in wherein there should be some specified end users you know who will say that, okay, this knowledge what is coming into the knowledge Based is proper or not. So, we should have some kind of approval process in place before using this knowledge base or before excluding this knowledge base for cleaning. So, let us summarize we said that good data and bad data are generated by end users.

So, we should give power to the end users to correct it or to clean the data and by doing so, our work of exercise will minimize second percent said that the way induces correct data or any person corrects data is by using some kind of a knowledge base which is improvised, improvised and improved and it keeps on growing as time goes by. And the last thing they said that they also need to ensure that this knowledge base is good. So, we should have some kind of approval mechanism in place. So, we need these three things right to be followed when we are cleaning the data. So, if we are giving this power to the end user, if we are creating any kind of have to than the tool should ensure that these three things are followed. And that's what exactly dq s does.

Dq S is nothing but it is a tool, which the end user as well as programmers can utilize to clean the data. And the way the dq dq s works, it uses a knowledge base. So we can see that dq S is a knowledge based tool to clean data, which comes in into a system. So the first thing we need to do is we need to go and enable dq s into our system. So to do what we have to do is we have to go to our SQL Server. And over there you can see that there is something called as data quality services.

And we need to go and install dq s. So by default, dq S is not enabled in msbi, right, so you can go and click on this data quality server installer. So once you do that, if you click on this, a small DOS prompt window will prompt like this. You can see and he will Say that Okay, now I'm going to go and install it. Now what does this installation mean? What does it do at the end of the day, this installation actually goes and creates certain databases in your SQL Server. So you can see that when you start the installation, he says that please enter a master key for your database.

What is this? Now remember that at the end of the day, we said that it is a knowledge base, the tool is not important, that knowledge base what the end user gathers is important, right? So you'd like to go and protect this knowledge base as well. So this password and key is nothing but it is. It is that thing you know, by which when somebody says that, okay, I want to go and use this database, this key without this key, he cannot see the database. Okay, so I'm going to go and enter a password.

So this is just for security purpose, to protect your knowledge base. So we're going to enter a password so this installer will take this password and it will go and create three databases. I'll talk about that. So Let us go and press Enter confirm it Yes. You would like to continue Yes. So, now you can see that the installation has started, this takes like five to six minutes and sometimes it even take 10 to 15 minutes right depending on how your system is.

So, let us let it execute and then we will go and see that what kind of databases it creates at the back end. So, once the dq s installer is completed, you should get a message like this saying that dq s installer finished successfully and in case you're not getting this message, you know, then there was something seriously wrong in the msba installation right. So, once the installation has finished, now, let us go to the database and let us see those databases. So if you see here, so you can see that I'm seeing the SQL Server Object Explorer. So the first Thing is, in this Object Explorer you can see lots of databases. But whatever it is with the name dq s underscore, these are meant for dq s. So these three databases here dq s mean dq s projects and dq s staging are dq SS databases, okay?

So, the first part in our dq s is the knowledge base. So the knowledge base actually gets stored into this dq s mean it is the most important databases of all of these three. Second is once you have the knowledge base into your dq s main, you'd like to use that knowledge base and you know, run it on your data right. So all these things are done by using this dq s projects. So all those things are stored into this dq s projects. And finally, you know, when you upload data from Excel sheet, you know from CSV, they all come into this staging data here.

So the staging data is, you know, is a place where it actually takes in the data temporarily and then does massage on it. So dq s main for knowledge base dq s projects, you know, when you start utilizing the knowledge base, you create a project etc. and dq s staging is where the data comes in into SQL Server. Also, I would like to highlight a very important point here, it is very rare that you would be dealing directly with these databases. In other words, it is very rare that you would go to one of these tables and do something, okay. And I would suggest you to not touch these tables manually or do something because it is maintained by the dq s system.

Wondering why I'm informing you about these three database, let us see that tomorrow. If you want to move from one server to another server, then you should know which databases you have to shift right. So basically, the only thing why I came here to inform you about these three databases. Tomorrow when you are doing a migration from one server to another server, you have to migrate all of these three databases because your knowledge base your project your data, everything goes into these three databases, or it is very rare that you would be going around and doing some You're manually and don't do that, because if you do that, then you know you'd be fiddling with the dq system. So just for your information, I've said that these three databases are there. And most of the time, what you will be using is the dq s software or I'll say you will be using the dq s client.

So you can see that we have a data quality client. Remember, in dq s, we have only two things. Only we we have the data quality server installed, installed, which we already ran. And the other one is data quality client. So this software will actually operate on those three databases and do things for you. So do not touch those databases, whatever you're doing do with this software.

So I'm going to go and click on this data quality client. So that is opening. So the first thing the deaconess client says is, tell me which is the server where those three databases are. You can see that at this moment, I'm demoing reading this by using SQL 2012. But whatever steps I'm seeing here is valid for SQL 2014 as well. So there aren't big changes between 2012 and 2014.

And why do you see 2012 here because in this PC, I have both the SQL Server installed. So I have SQL 2012 as well as having school 2014. And I think when I ran the software, I ran it through the SQL 2012 the decreased installer so it has installed in 2012 at this moment, so that is my mistake. But you know assured if you are in 2014, you should get you know, such kind of software there as well right. So you means you know over here what has happened is I clicked on SQL 2012. So I went here and I went to the data quality services and I used here, what you have to do is you have to use this SQL Server 2014 and you will find a similar amount over here.

Okay, so whatever steps I'm showing here is also valid for 2014. So let us go and connect to this server because that is the server, you can see that that is my server name. So that is where the databases, all those three databases. So let me go and click on Connect. Now using dq S is a three step process. The first thing is you create the knowledge base.

The second thing is you go and you execute the knowledge base by creating the creating a project. So in this second thing, you know, what we do is we take that created knowledge base, and we execute it on a data, right. And the last one is we administer, we monitor, you know the progress of the project. So creating the knowledge base using the knowledge base, and monitoring the knowledge base. So these are the three sections. So let's first start creating a knowledge base, right?

So I'm gonna go and click on the new new knowledge base menu here. So let's click on your knowledge base. So let's say that this knowledge base what I'm creating here is saying steamer cleaning. So this is a knowledge base, which will actually help me to clean my customer records. So, we are going to go and clean these things what we have here, right? You can also use the existing knowledge base.

So in case you have existing knowledge base, you can also use that and also you can import from a decrease format file your existing knowledge base. So at this moment, I will say none I want to create it fresh. And I will say that this is a domain management. And I'll say next domain your means nothing but business rules, business rules, you know, which will define your cleansing process. So, in order to clean this customer file over here, what kind of rules you want to run? Right?

So that's what exactly domain here means. So I'll say Okay, first rule, I want to run on the product name, right. So let me go and add a domain and I will say that This is product knee, right? The type is string and you can see that you can also enable speller spell checks here. So I will say that is enable don't enable the spell because I would be creating this domain myself manually. So don't enable English check on it and I will say OK.

So, you can see that now I have created a domain here called as product and for that we need to go and define rules, we need to go and define values and a lot of things. So you can see that we have five tabs here, which will help us to define the cleansing process or the cleansing rule. So let's first start with the most important tab that is term based for relation. Atomic distillation is nothing but you know, you have to specify the rule which says that for which value, you want to correct it to what means what, for example, in our Excel sheet, if you see we will Want to correct the clothes that is similar to clothes? Right? So you will go to the term based correction rule and can see that there is a plus sign here, you will click on this plus sign and the time you click on this plus sign it you know, you get a small data entry tab here.

I don't know why it is saying in process I don't know why it's so slow, right. So you're in a closed needs to be corrected to close. Close. So this is you know how you build your knowledge base right. So again, I would like to go and add one more relationship here. For example, here I will say this item code 98 is nothing but actually close.

So this is close. Again, let's add one or two more rules. For example, this Reebok here is nothing but shoes. Right. So, this is shoes and what else we have Adidas is nothing but shoes. And this World Cup around here in India in Hindi, it means close.

So this will again change to close, right? So I've made all my corrections over here and also you can go and apply domain rules, in other words validations. So, you can see that you have a domain rule Rules tab here. So you can go and add a new domain rule and it gives you hordes of validations you know, where you can go and check if the length is equal to this of value contains this, you know, it matches this particular expression and so on. So you can see that there are lots of validations here, what we will do is Let us go and put a regular expression. So we'll say that if the value does not match this regular expression what I'm going to write then you know the rule face.

Now in case you're new to regular expressions, you know what my suggestion is to go and watch the regular expression video in the dotnet section where I explained how to write a regular expression. But here what I would like to go and put a validation is that when you say product name, you know when somebody says product name, I don't want people to type in this numeric characters. For example, you know, I don't want people to go and say that okay, this is a product name right, because product name has to be characters it cannot be numeric. So, what I will do I will put a regular expression here. So I will say that, in this a person can only enter A to Zed, so you can only enter A to Zed and also let me put the maximum and the minimum characters. So, let us say that we will see that this product name cannot exceed more than 20 characters.

So, I'll say that minimum it has to be one character and the maximum it has to be 20. So, regular expression is nothing but it is a pattern matching technology in a wherein we go and we put an expression and the validation happens accordingly. So, I put a regular expression here saying that the person can only enter from A to Zed characters and the minimum is one and the maximum is 20. Now, you can see that we already have a value here called us item code 98 right. So, for this value also have I have defined the mapping So, you can if you remember we need to we need to put a rule here So, I'll say that check on numeric or numeric right. So, this is one regular expression validation what I put here now, if you remember in the Based a validation it contains invalid data check numeric description, check numeric scheme is checked numeric right.

I don't know why it is giving I think it was because of the name right. So, I put a domain rule here you know which checks for numeric characters right. But in the term based relation, you know I have said here that this item code 98 will be replaced by clothes right. So, let's let us go and delete this because this one will contradict with this domain rule right. So, let us go to this domain rule here and delete this sorry, let us go to the term base relation and delete this rule because this rule will contradict with our domain rules, right. So, in short I put a domain rule you're saying that you cannot enter a numeric number.

It has to be repeated Your characters and in the term base relation are defined you know what corrections should be done, if a value is found the third tab what we have here is the domain values, domain values are like tonus relationship you know, but it is more at a global level that means, you can use these values in other projects as well. For example, these Tommy's relation what you see over here you can use only within this project, but the domain values you can use you know across a lot of other projects. So, at this moment you know I have one domain value here called is dq s nl. So, he says okay this is the QoS knowledge Do you want to connect it to something? So, we will see when we run the project, we will give this value during the runtime okay. So, we have defined term based relationship we have defined domain values we have defined domain rules.

There is also something called as reference data. reference data is nothing but you know when you want to refer data from third parties For example, now, let us say that you want to do English check by using some dictionary service. So, you would like to get that data from the data market right and check it over here. So, in case you want to go and take data quality also data services from some data provider you can use this. So, at this moment will not use this you know, but you can easily go and browse here and attach online data service provider and then you can again you know, you can again see that okay for this value I want to have this value, right. So, domain properties is where we define the datatype.

Domain rules is where we define our validations Tongass relationship is where we define that for that value what it should be corrected to domain values is nothing but constants but more at a global level. And reference data is for third party sites, right. So, everything is done We are going to go and click on Finish. Once you click on Finish he says that do you want to go and publish this? Complete the the knowledge base, you know, so that somebody can use it. So I'll say yes publish it.

So once I do publish, you know, he will actually go and store this knowledge base into SQL Server. Right, we can see down below, I can see the customer cleaning knowledge base. And if I ever want to go and again open, I can always click on this small link over here and then click on domain management. And I will again go in edit mode. So you can see that again, I'm in edit mode, right? Great.

So we have created the knowledge base, now it is time to use that knowledge base. So let us go and click on this new data quality project. And we see that okay, test our Excel and down below, you can see now that our knowledge base has popped up. So I'll say is use this customer cleaning knowledge base. What I've created And I want to do cleansing activity at this moment. So I'll say next.

So now the next thing is I'm going to go and choose Excel here, because we have our data in the CSV file, and we provide the source, you know, the source data on which we want to run the cleansing process. Now, the next thing is we need to go and map the columns with the domain rules. So if you remember at this moment, we just have one domain rule. And we have two columns. So we have not created any rule for the sales column. But we have created the rules for product name, right?

So select at the left hand side, the column name and then select on that column, which rule you want to run. Right? So I want to run the product name domain rule on the product name column. Then I will say next, and then we click on Start. So once we do a start, here it goes and it takes those domain rolls and start running, running on the column data. So you can see that you He showed me some statistics here saying that there are two invalid, invalid records and total nine records have been passed.

I've also amended the data a bit. So you can see that there are total nine records. And in that I have put some Numerix, put some nulls and all those, right. So let's go and start analyzing, you know, what is the output so I'm going to do a next year and let us try to analyze what kind of output is showing. So first thing is go to invalid. So it says that in invalid if you remember, we had said that we do not want numeric values.

So you can see that here it is showing me this value. And it is showing me that this failed numeric value, check numeric, right. So I'll say that okay. This is 909890 please correct it to close, because probably this is a code for quotes, right? And I will say approve. If you remember, I said said that at the end of the day, whatever you know cleansing is happening it has to go through an approval process, because we have humans involved and again they can make mistakes right.

So, I'm saying yes, I approve that or I know that this 909890 is nothing but core code for or as a protocol for close and I will say approve right. So, that it is approved. And you can see that once I approved it, it goes into the corrected tab. So, in the corrected tab, it shows that the 909890 will be corrected to close right. Also you can see that there is something invalid here because if you remember in our records once the record is empty, so it is showing it is it is saying here that this this record one of the record is matching ridiculous null constant right the value. So I will say is correct this to suspense account because I do not know you know what kind of product is there right?

So you can see now I have corrected a number to close I've corrected you know the knowledge to suspense. Now let us go on this new tab here and this new tab is now giving me term based suggestions. So if you remember we had said that Adidas his shoes, clothes his clothes then if you go down below we said a couple of his clothes and then Reebok his shoes. So you can see over here he has gone ahead and started searching in the term based relation and you can see this number here saying confidence confidence is that I'm hundred percent confident that you know this Adidas issues because exactly the value is matching. So what I will do is I will say that okay, whatever is confident or is from a term based, I will just start approving them. So you can see that I'm approving whatever is hundred percent matching and the rest of the things I will not do approve, but you can see that we have this item code here item code nine eight.

So, I will say that please correct this to close because this is a this is a color this is a code of clothes right and i will say approve and I will say next. So, you can see the final output preview what it is showing here, the most important column here is this modified by So, it says that, these are the values you have modified and you can see the previous values you just named this as new new value saying that this has not been modified. So, wherever you see modified by user that means by us It means that that column or that value I will say rather has gone through a cleansing process. So, if you remember we made clothes clothes, we made this item called clothes, we made Reebok Adidas shoes, we made coupler as clothes, we made the number as close and we made the null as suspense.

So we have corrected you know, all these values and the first two values we have not touched. And after that, you can go and output this value into SQL Server if you wish or if you want you can output to to a CSV and also it gives you options saying that do you want to just output data or do you want to just output the data as well as the cleansing information? So, what I will do is I will say that I just want to output data and I will say I want to output into a CSV file. So, I will go and create a simple CSV file here saying cleaned output right. So, this is a clean output and IBC export that it has exploded. So now let us go back to the folder.

So if you now go to the folder to just check the cleaned output, so cleaned output. Next Next, so that it is Oh, it is a comma separated fine, right? You can no I hope that because I tried to open an Excel and I did not choose the proper values let me go and open again it's not opening now properly okay whatever So, the point here is okay here it is again it is coming in. So, let us say this is delimited This is delimited by a comma characters such as comma tab separated isn't that good next, it is not tab separated it is comma separated next and say finish. Yeah, so, there we are seeing the output properly. So, it says now, you can see that all the values has been corrected.

So, if you remember, this was our bad data, right So, you can see here, Claude has been corrected to close. This is close the disclose item code has been corrected to close Reebok and Adidas has been corrected to shoes copra has been corrected to close and the non null value has been corrected to suspense. So now we can see that we have a clean data over here. And the last step here of dq S is monitoring. So, you can click on this monitoring and you can see that how much time the cleansing process had had, how much time the cleansing process has ran. So, you can see that I have been running a lot of cleansing process.

And you can also see that how much time has been elapsed. And you can go to every one of these cleansing activity and you can go and see that what actually happened in each one of them. So you can see that how many records were corrected how many records are suggested and so on. So in a nutshell, dq S is a three step process. The first one is you create the knowledge base. Second one is you create project and you use that knowledge base and the last one is you administer your activities.

So, you can see that how this dq S is so user friendly. Any simple end user can use This. But now think about it. If we developers can use this knowledge base, what is the end result of this dq s, the biggest end result of the take us is the knowledge base, the knowledge base, what the end user is entering into this knowledge base system. So if we can use this knowledge base into SSIS, that would be really great. So if we can have some kind of a competent exercise where we can use this knowledge base, a lot of things can be minimized, right?

So let us try to understand that how we can use this knowledge base system into SSIS programmatically. So whatever I've shown you is a tool, a tool, which you can go to the end user, and the end user can solve this problem and solve our problem also. But now think about it, that we leverage this knowledge base and we take into SSIS. So my next demo is that how we can use bicurious service into SSIS problematically. So, let us try to understand that how we can use this knowledge base into our SSIS package. So, let me go ahead here and add a new SSIS package and I will name this SSIS package as cleansing or something, right.

So, there it is so, let me rename this package and I will name this package as cleansing right. And let us go ahead and add a SSIS data flow task so, I'm going to go here to SSIS and add data flow toss and I will name this task as cleansing and cleansing. Okay, that's so wrong spelling clean Singh Now in the SSIS package, the first thing is we need to go and read the CSV file, or we need to read the source file, right? So I'm going to go and read the Excel source file. So I'll say this is the bad data that comes in. I recently added a new I will browse to the bad data what we have and we'll take the sheet one.

And yes, I'll take all the two columns and say okay, and this input, I will give it to my component, so I'm good. So we have a limit component here. So if we go here, we have a ready made component called as dq s. Where is it? All those sorts of source of sorry non other sources dq s cleaning. So, this bad data will go to dq s cleaning and this dq s cleaning will use our knowledge base. What is that into those databases and do the processing.

So I'm going to go and right click here and say Edit. And over here, the next thing he asked me is asked me is that where exactly is your dq s database? So, I will say that yes, my dq s database is on to this server. So let us copy this Ctrl V. And from as soon as it goes to the server receives that okay, I can see a lot of knowledge base in that Which one do you want to select? So I will say that I want to select customer cleaning and to this customer cleaning out the map, my domain product name. So basically to that rook Name field I want to map this product name.

So, the product name column I want to map the product name domain and I will see okay and this whole output of the dq s I will give it to my my CSV or some kind of output. So, let us go ahead and just create a destination here flat file and over here I will say that this output let us go ahead and how he said this is massage data or I can say corrected data. So, I will go ahead here and I will say that I want to create a new file here, a CSV file which is a corrected data right and I will take all the call ups as it is So I will see okay. So I can see that the source output the status, let me see let let us see that what the output is, and we'll see okay. And I will go ahead here and over here let me go and enable the Data Viewer to see that what kind of output this dq is cleaning is sending me and let us go ahead and run this program.

So I'm going to go ahead here and set this as the start of file and let us run this program. So once he runs the program and you can see that the Data Viewer output is showing me the output which is applied after the knowledge base is applied. So you can see that he had changed the float to floats. Here change the Reebok to shoes here change the Adidas to shoes, he has changed the color to shoe so you can see that basically he's applying that knowledge base which is If selected yes definitely he does not give me an interactive UI you know very very I can see that okay this item code is close right. So, that kind of interactiveness we do not have, but you can see that he is applying the current knowledge base and giving me the output. So, if I go and run this and if I go and see my final output so, if I go and see my file, my file at this moment is corrected data.

So, if I go and see my failure very quickly, can we close the program if I see my firing out the file name is corrected data. sorted a school year corrected data so, if I go and open this file in Notepad I should see a file which is cleaned Which is cleaned after applying the dq s. So, let me go and open this and there it is. So, you can see that basically he has applied dq s and output is seen. So you can see that he had changed the clothes to clothes, he had changed Reebok to shoes, he had changed Adidas shoes, he had changed coupler to clothes. So you can see that basically, our program at this moment is utilizing the current knowledge base given by dq s and improving the quality of the data. So this dq S is a service.

It can be used by end users, it can be used by programmers. And the best way is that it can bridge the gap between the end users thinking and programmers. So for for us, you know, if the end user starts collecting data, we can use that knowledge base and reduce a lot of our work. I remember that In the prior launches of SSIS, you know, in 2008, as well as in other versions, I remember that I have tried a lot of code to achieve this. But as soon as end user starts participating in improving the data, then the life of the developer becomes very easy. So I think this is a wonderful tool.

So either you can use it standalone for the end users. Either you can use it for the developers, or you can use it for both of them for developers, as well as end users. So the end user can keep on updating the knowledge base. And developers can use that existing knowledge base and start reducing their work and start making that SSIS package more efficient. So I hope that you enjoyed this session. In this session, we were trying to understand what is the importance of dq s and how it can be used by end users as well as programmers Thank you so much.

Sign Up

Share

Share with friends, get 20% off
Invite your friends to LearnDesk learning marketplace. For each purchase they make, you get 20% off (upto $10) on your next purchase.