So, welcome to lap five this lap five is a 30 plus minutes of training and in this 30 plus minutes 30 plus minutes of this training you know we will try to understand what is Cosmos DB What is this thing called is consistency levels in Cosmos DB then we will go to the Azure portal and see that how we can create Cosmos DB how we can create documents how we can create databases, collections and so on. We will also look into that how can we handle failover So, if there is a fail or how to handle it, and then finally, we will see that how we can use C sharp to connect to as your Cosmos dB. First thing you know before I start this tutorial, let me tell you in 2014 around we had something called as document dB. So, this Cosmos DB is an improvised version of the document DB okay.

So, let us start this tutorial. So, before we start anything else, let us first define what is Cosmos dB. Cosmos DB is a planet scale No SQL JSON database with multi API support. Now this definition looks long, you know. So what we'll do is first let us try to define this word cosmos. Cosmos means universe, or to be very specific, it means a well ordered universe.

It means something big. And that's what Kosmos DB intends to do, it intends to give you a planet scale database. So, now, let us first try to understand what this planet scale word means, and what type of problem does this planet scale database tried to solve? So here is a situation you have your database somewhere in us. And then there are users who are using this database across the geography. So when any user who is in the US location accesses this database, he gets the data faster as compared to the users who are in other geography like India, Nepal, UK also so on.

So the performance For the users other than the US location is not up to the mark. So, the solution for this is to bring the database near to the end users geography. So, let us say if I'm from Mumbai, then I would like to access data from the Mumbai data center. So in other words, you know, what we will do is, you know, in whichever places we have our users, you know, viewed set up their secondary databases or read database and our primary database or we can also turn that as the right database is in the US. So when someone writes in this us database, we will send out replication to databases which are in other geography. Now please note, do not compare this with CDN or, or BCP.

Because CDN, BCP dr is a byproduct of Cosmos dB. But the main intention here is to bring fresh data live data near to the user's geography. In case of Dr. And BCP, we do not read the data from the database, it is only used you know when there is a failure in the main system. While over here in case of Cosmos dB, we are going to go and read database from the secondary databases. So do not confuse this with Dr. And BCP plan. So again, I would like to clarify the vocabulary, the main database, you know, where the data would be written, we will term that as primary database or as a right database.

And the other databases we will term them as read databases or secondary database, because this is the vocabulary I'll be using henceforth in this tutorial. Now, one of the biggest problem to achieve a planet scale database architecture is consistency. So here is a diagrammatic representation of what exactly is a consistency issue. So now let us say you have a US database, your right database, and from this right database, you are actually single single into a Indian database which is in India. So, now, let us say if if you write something to this us database must be takes one or two minute to replicate this data to the Indian database. So, at that moment before the synchronization finishes, when an end user reads data from us, he would see a different data as compared to a user who reads from India.

So, in other words, at some moment of time, you know, when the synchronization has not finished, the data would not be consistent. Now, this is termed as eventual consistency eventual consistency means that we do not do anything special as such, we just leave it to the network, we just leave it to the transactions to the locks of whatever we are having at this moment. And because of this, at some moment of time you'd get in inconsistent data for the end users of different geography Now sometimes this consistency is okay you know if your application is not mission critical and so on must be that this is okay. But now think about that Do you have a stock market application. So, in case of stock market, if the values are inconsistent, then it then it would really create problems. So, for this issue, we can switch to a more hardcore consistency called as a strong consistency.

In strong consistency plan the read clients will not be allowed to read the data which is not committed to all the geographical read areas means what you can see that here is a small diagrammatic representation of the same. So, this is this P stands for a primary database. This S stands for one of the geographical secondary databases, and the s two is again one of the secondary databases. C two is a client and C one is a client for the respective secondary data database okay. And you can see here you know, data is getting committed on regular intervals on the primary database, okay. So, let's assume that first thing there is a value is equal to A there is a value at 9am then at 10am somebody commits value B into the primary database.

So, the primary database as soon as the value B is committed, he starts doing the synchronization. So let's say at 10am itself you know the synchronization message is sent to Bob is sent to both of s one and s two. So the value B is now they have started synchronizing the value B two s one and s two. So at 10:15am, the s one sends a message saying I'm done with the updation and let's say at 1030. s two answers also says that I'm done with this information. Now remember that in as your Kosmos dB, it does not really take half an hour to update the database. This is just an example.

So please note that this happens literally in seconds, okay, but I'm just giving it as an example, so that you can understand it. So, in other words, the value B which was committed in the primary database is committed to s one and s two at 1030. So, at 1030 it is confirmed that everything is okay. So, now, what the primary database does is once it gets acknowledgment at 10:35am he says to everyone that everything is successful, right. So now, until this 1030, if not, let us say C two tries to read the data, he will see the value a and not value B. Okay.

Even if c one tries to read the data, let's say at before these commits happen, he will also see the value a not only he but if Even somebody tries to read the data from the primary database, he will also see the value a. Now this is consistent. So, even if the value B is committed at 10am, but all the clients who are reading from any one of the database either this primary either it is secondary, they are all seeing a consistent value, okay. Now, when the value B is committed to all the database that is approximately at 10:35am right after that, if somebody tries to read the data from primary he will get value B. Even you can see when c two tries to read the data, he also gets value B and C one also gets value B. So, in a strong consistency, we do have latency.

We do have performance issues, but it is consistent. Any client who reads from any geographical area at that moment we'll see the CSM data Right. So this is termed as a strong consistency. Now, both of these consistencies have their own trade offs. So, if you're selecting eventual consistency, you have low consistency. So, at some moment of time, all the clients who are in other geographical areas in different geographical areas can see different data right.

But then you have high performance because nobody waits for anyone. In case of strong consistency, it is highly consistent, but then you have performance issues, you have latency, latency means because you know, all the clients will not be seeing any kind of uncommitted data. So, any database gets committed on the primary database, it takes some time to propagate to all the secondary databases and till then all the clients see stale data. So over here, you will be having latencies you will be having performance issues. So both of them have their own strong points in both them have their own negative points. And the choice should be made as per your application criticality.

If consistency is playing, then you should be selecting strong. If performance is important and consistency is not trying must be, you can select eventual. But wouldn't it be great if we can have more consistency, or we can have more flexibility, which we can select between strong and eventual. So, between these two extremes? If I can have more choices, then it would be great, right? So that's what in what in as your Cosmos DB what they have done is between the strong and eventual, they have given you more three choices of consistency.

So one is the bounded consistency, other one is a session consistency, and the third one is a perfect one, right. So let us understand these three as well. One by one, bounded stillness in bounded stillness. We will see data, which is some hours old For example, I can go and I can set the status to two hours in bounded stillness. So, all my clients will be lagging behind the right database by two hours right for example, let us say that you have committed some data at 11am s value equal to a and at 12 you have again committed value equal to b. So, even after the committing of this after 12 the value equal to b still everyone is reading a.

So, you can see when a client reads from the secondary database, he still sees the old stale value of a even if somebody is reading from the primary database, he also sees sees the value of a right. So, this is termed as bounded stillness. So, in bounded stillness, there is a lag and and you're okay with this lag, but the data is still consistent. In other words, I think Anybody reads from the primary database or any other secondary database, they all see the same value. If you set the bounded staleness time is equal to zero, then it becomes a strong consistency. Let me again repeat that sentence.

If you set the bounded staleness time as equal to zero, then it becomes a strong consistency. The other important consistency is a session consistency in session consistency, or whatever it is, or what other geographical databases how much they are up to date. So, for example, I look at this, you know, here we have the US database, it takes two minutes to synchronize to India. So, readers who are reading from the Indian database would be following the eventual consistency. But the users who are reading from the US database means from where the commits have happened, right, those users would see the recent data. So over here it is kind of inconsistent, but for the user who is Writing the data to the database for him it is consistence because whatever he has committed he can read it, okay.

So, this is more from the perspective of the clients who are committing the data that they should see the fresh data, but for other users you know even if they see the data with the eventual lag or with the time lag It is okay. The next one is consistent prefix, inconsistent prefix the data is seen with the same order by which it has been committed to the right database. For example, now, let us see on the primary database the value was set as ABCD. So, when So, and then later the synchronization happens or whatever the time it is, five minutes, 10 minutes right. But the data when was read by the client, they would see in the same sequence so the order will be maintained. So, remember between the strong consistency and eventual consistency, you have three more choices.

So remember first in strong consistency, everybody sees the data same as it is, while in eventual consistency, you can have things which are up and down, but then none of the clients will be waiting. bounded consistency is having some kind of a time period or some kind of a prefix. In session consistency, the client who commits the data will always see the fresh data, but other people or other clients who are in other geographies areas can see stale data, depending on the time which it takes for application, then we have prefix in prefix, the sequence by which the commits have happened in the same sequence, it will be read, right? So now you have the flexibility to choose between strong and eventual and to make more better choices, right. So in other words, you know, when you say that you want to make a choice of a consistency.

It really depends on two factors that if you want To be strong consistent, or is performance the criteria. For example, you can see if you say that you want to be strong, consistent, right? Then please note your performance will be down, you will have latency, right. But your consistency will be high. But if you say eventual, then eventual, the consistency would be low, but your performance would be higher availability would be high. So whenever you're making choice of these consistencies, you should be very sure in what your goal is, what kind of database you have, what annual application how critical it is.

If you remember, 10 minutes back, I gave the definition of Azure Cosmos DB as a database. It's a planet scale database, which is stored in JSON format, and it has multi API support. Now, what does multi API support means multi API support means that the that the data is stored in JSON format, but you can use Any API which you're comfortable with, for example, I can go and say select star from table name, right? So I can use an SQL API to get this database to get this values, right. Or I can use a Mongo API, I can use a graph API, I can use a table API, I can use a Cassandra API. So So the other good thing about the Azure Cosmos DB is that he does not tie you up with the type of API you want to use.

Right? He gives you multi API support. A lot of times I've seen people saying that, okay, this database is supporting SQL, you know, because there are joints that are this that are that could create, right? So that's why you store in SQL Server, then you say, okay, some of the data I will store in MongoDB, because you know, we have key value pair, and I can just look up. So you end up with having different databases at the back end. No, because of the nature of the data, or whatever it is, right.

So over here. Now, you can go and you can choose saying that I want an SQL API or I can add, I want a Mongo API or whiskey. of API one. So remember, Kosmos DB is a planet scale database. It's a planet scale dB, which is like no SQL. It stores data in JSON format and has a multi API support.

So that was a 20 minutes theory on as your Cosmos dB. And I think it was worth investing time on the theory part because the consistency is so confusing that many seasoned as your administrators do not know which consistency to choose, and they end up paying heavy bills, right. So that's why I spent my 20 minutes of time explaining you the different type of consistency. So if you want to go and add as your Cosmos dB, you can see at the left hand side, we have a manual, so you can either click here, or either you can click on the plus new and you can just type your cosmos. So as your Cosmos dB, there it is. Right and I'll click on Create.

So as soon as it Click on create it says first thing like, give an account ID this account ID is nothing but a unique name by which your Cosmos database will have an URL. So I'll say this is my Cosmos 123. So you can see now my URL will be my Cosmos 123 dot documents.zero.com. Okay. If you remember while I explained the theory, I said that as your Cosmos DB supports multi API support, so you can go and select you know, the API which you want here. So at this moment, I will select SQL.

Again, the subscription is pay as you go. I do not have any kind of pre created resource user group at this moment. So let me create a resource group here, race cosmos, so in this I will add all my Cosmos databases. And you can see now this is the most important part. So we are saying that our right location will be central us, and I'm saying enable geo redundancy. So this We'll actually go and enable Kosmos DB on the various other data center locations right so let's go and click on Create and you can see now as your Cosmos DB is creating can see this small sign over here indicating that as your Cosmos DB is getting created.

So we can see deployment in progress. So let us wait for some time until the cosmos DB gets created. So you can see now there the deployment is successful and it says that the document DB now remember as we said, you know as we started this video, I said that as your Cosmos DB is evolution of document dB, so you can see here it says deployment of document DB is successful. So if I go to now to as your Cosmos dB, you can see that the resource has been created. So let us go inside this resource. And let us say I want to go and create a database of cosmos.

Now the structure of Cosmos DB is as follows. At the top we have database then we have a collection and then collection has the Jason Right. So, let us click on Add collection out here. And so, you can see in this add collection. So, that is going put some database name here DB one and inside this DB one I will create a collection called as collection one call one I will not say Unlimited, I will say fixed remember, always be be a miser, right Do not try to select something which is at the higher end, right. And and I'll just go and say, Okay, all right, you can see there is something called as an ru.

I'll talk about that later on unique keys and so on. We'll talk about it later on. And let's press OK. And let's try to deploy this collection. So that you can see now the collection has been deployed. And inside this collection, I can know I can now go and add items, right? So I can go and add JSON data inside this.

So remember, at the top we have database, then we have collections And then we have documents. So now I can go and click on new document. So you can see that by default it gives me an ID. So if you remember I said everything what you store inside Cosmos dB, aka document DB is a JSON data, right? So I can say the ID is 1001. And then I will say the name is Shiv light.

Remember, you can have different different JSON structure row wise so it's not necessary that you have to store in the same structure that's what the flexibility of no SQL is, right? So you can see I'm going and saving the first record the first document, so that it is 1001 I can go and again, click on New Document I can say this is 1002. And I can see now this has name Shiv and this has one more property are called as country. So country, India, right. It's a safe So we can see now look at the structure and the top we have database, then we have collections and then we have documents and every document is you know is nothing but it's in it's in JSON format right. Now, one of the other things you know, which we discussed in our classes.

So, we discuss about consistency, right. So, this consistency is set by using this replicate data globally. So, you can see the small menu out here replicate data globally. So, you can click on that and now you can see this nice screen out here and it says that So tell me, which are your right regions and which are your read regions. So, remember I talked about right region and read region, right? So I can go here and I can say that Okay, so at this moment, you can see it says, Your right region is central us and the read region is East us, but I do have the flexibility to go and choose you know what I want as retreat And propagation will start happening automatically.

Okay. Now please note that the name is planet scale as well as the money is planet scale. So So be very choosy in what you're doing. If you are choosing too many locations, you will be ending up with a heavy bill. So it's very important that which locations you want to replicate right. So there is one right region and there is one read region at this moment.

You can also see this default consistency over here. So you can see that the consistency which we discussed in the previous 20 minutes of the video, so you can choose strong you can choose bounded staleness, you can choose consistent prefix. At this moment, I will choose eventual, I will choose eventual because I do not want to spend too much money, right? It's a demo. But you can see like for example, if you put bounded staleness it gives you saying that tell me the minimum time the lag should be if you put session and consistent prefix, so You can choose any one of them, I'm going to go and choose eventual so that I get a less bill replicate data globally in that I have just chosen one right location and one read location. So, you can see now it is updating my default consistency here.

So there you can see that my default consistency is updated. Now, let me go back to this replicate data globally. You can see at this moment there is only one right region if you wish to have failover failover This means that if this right region is unsuccessful, then you would like to have another read region to become a right region. Right? So for that, you can go and click on this manual failover or you can go and click on the automatic failover right so whichever you want to select. So for example, let's say I go and on this automatic failover.

So it says that so go ahead and say like, for example at this moment, I have just one read region, but I can I can I can go and say that when there is a failover into this right region, then make this read region as a second priority. So, over here for example, now let me go and select let me go and say okay, let's say that I have different right regions here. So, let's say I have one more right region right. Let us say I will just save this please note that the more regions you select, the more bad it is you will get heavy bills. So this is only for the purpose of learning right as soon as the learning finishes I go and delete all my resources, which are connected to such kind of behavior built right. So you can see it is updating.

Let's give it some time and then I will go to automatic failover. So in automatic failover, then I will see two read regions. So I can say okay, that if there is a failover in the first right region, then I can go and give priority to my read region and say that Which of the read regions should become a right region? So let us see that option. It's updating remember, as your Cosmos DB is a planet scale database, right? So when you are upgrading any settings, you know, he has to replicate all across the geography.

So that's why it takes a bit of a time. So there, my regions have been created, you can see it's clearly a note here, each region is billable, you know, not only on the throughput, but also on the storage. So even if you're not doing anything your meter is going on. So please ensure that you make appropriate choice. So I'll say enable automatic failover. So in other words, if this right region fails, I want East us to be the first priority, and South Central us to be the second priority.

But if I want, I can drag and drop so you can see here, I can click on this small dot, you can see there's a small dotted sign out here. We can select this, why isn't it getting selected? On the left to reorder the list? Yeah, I'm trying that. So click on this and there it is, ah, it's horrible what's in the UI? Why?

Drag and drop the read regions to reorder tip drag this on the left to reorder. Okay, so I'm dragging this to the left to reorder. Oh, that it is it's done now. So, whatever it is, you know. So, you do have the option to go and change the orders from here right and you can go and switch to automatic failover right. So, remember, not only the read but also if you want you can make the right regions as well as automatic failover enabled.

Good. Now, the next thing what we would like to do out here is we would like to go and use some programming language like C sharp or something you know to go and connect to this as your Cosmos dB, right. So let us see a small example. How to go and connect to as your Cosmos using shisha programming language. So in order to connect to the cosmos dB, you need the keys. So you can go and click on this key section out here.

And from here, you can go and choose the keys, right? Or you need to use these keys in your sharp language. So you can see the code of C sharp looks something like this. So first is the endpoint URL. So the endpoint URL at this moment is my Cosmos 123, dot documents, whatever, right? So I'm going to go and connect over here that I need the primary key.

So I'm going to go and choose my primary key. I can copy it right. And I can put it right out here. So there it is. Okay. And that's it.

Now remember, our online as your Cosmos DB at this moment has a structure of an ID and a name. So what I'm going to do is I'm going to go and stick to that. So we have have an ID and we have a name, right? So you can see what I'm doing is I'm creating first object of document client. And in this document client, I am passing the URL and the primary key. So this key will help me to validate to my OCR, and then I'm saying Okay, so we have created a collection here.

So if you remember. One is that you can go and you can create a collection if you wish, right? So you can see we have the collection, what was the collection name? So let us go to Data Explorer. Our collection name was DB one, call one, right? So DB one and call one I'm sorry.

So let's go your DB one, right. And the collection was called one byte. So in this you can see I'm saying, please go and get me. This x type. This x is nothing but the JSON class the JSON data, right? So you can see that I've just kept the same kind of a structure.

So please go and get me The JSON data of the as your Cosmos DB and we can then go and look through that and I would like to display the name ID as well as I would like to display the name right. So x dot the name okay. Right. So, at this moment, what I have done is you can see that I actually am removing that region. So, that region is still going on. So, once this region is done right, we would like to go and execute this program and we would like to go and see the output.

So there you can see that that region has been removed. So now if I go and run this program, I should see two rows, one is 1001 shift and the other one is 100. To shift so if you remember, if you go to a Data Explorer. So in the Data Explorer, if you remember we had two documents, right? So, we had 1001, which is having the value shift, that is 1002, which is having the value ship, you can see, there are a lot of other properties as well, but these properties are as add are added by as your Cosmos dB. Now remember that whenever I am taking these trainings at this moment of as you're right, I'm not concentrating too much on the programming API at this moment.

Why? Because I am planning to have separate topics to just go and query as your Cosmos DB to query as your tables to query as your blobs. So, you will see that in the later sections of the as your training, you will see a lot of C sharp with us here. But at this moment, my goal is that to make you acquainted with all these concepts of Azhar, which are very confusing, right, so, so at this moment, do not worry too much. You must be thinking that Oh, tell me that. How do you create a stored procedure?

Tell me how do you do that? Absolutely. I'll tell you, but at this moment, let us first try to cover the breadth of as your because as your is a huge thing right. So let us try to Understand what are the different options given Bezier and then programming is easy. We are shisha programmers, you know it's we have been doing C sharp for a long time. So, that is something we can do it more easily.

But these concepts like consistency levels you know as your tables, function ABS These are so new things for us that we first need to understand those concepts and then the programming part is easy. So good. So please note that we are going to have that in the later sections of the training. So that brings us to the end of this video. So in this video, we were trying to understand what exactly is as your Cosmos DB what are the different consistency levels you know, which are available in as your cosmos and how to go and use sharp to connect to as your Cosmos dB. Thank you very much.