Lab 8: Block blobs, Append blobs and Page blobs

Azure Step by Step Training Lab 8: Block blobs, Append blobs and Page blobs
55 minutes
Share the link to this page
Copied
  Completed
You need to have access to the item to view this lesson.
One-time Fee
$69.99
List Price:  $99.99
You save:  $30
€66.52
List Price:  €95.03
You save:  €28.51
£55.40
List Price:  £79.15
You save:  £23.74
CA$97.77
List Price:  CA$139.67
You save:  CA$41.90
A$107.48
List Price:  A$153.55
You save:  A$46.07
S$94.07
List Price:  S$134.39
You save:  S$40.32
HK$544.73
List Price:  HK$778.22
You save:  HK$233.49
CHF 61.85
List Price:  CHF 88.36
You save:  CHF 26.51
NOK kr773.61
List Price:  NOK kr1,105.21
You save:  NOK kr331.59
DKK kr496.19
List Price:  DKK kr708.87
You save:  DKK kr212.68
NZ$119.31
List Price:  NZ$170.45
You save:  NZ$51.14
د.إ257.07
List Price:  د.إ367.26
You save:  د.إ110.19
৳8,370.95
List Price:  ৳11,959.01
You save:  ৳3,588.06
₹5,916.42
List Price:  ₹8,452.40
You save:  ₹2,535.97
RM312.50
List Price:  RM446.45
You save:  RM133.95
₦117,624.49
List Price:  ₦168,042.19
You save:  ₦50,417.70
₨19,473.08
List Price:  ₨27,819.88
You save:  ₨8,346.79
฿2,426.48
List Price:  ฿3,466.55
You save:  ฿1,040.07
₺2,418.28
List Price:  ₺3,454.83
You save:  ₺1,036.55
B$406.67
List Price:  B$580.99
You save:  B$174.31
R1,268.60
List Price:  R1,812.36
You save:  R543.76
Лв130.09
List Price:  Лв185.85
You save:  Лв55.76
₩98,070.61
List Price:  ₩140,106.88
You save:  ₩42,036.26
₪261.37
List Price:  ₪373.41
You save:  ₪112.03
₱4,133.88
List Price:  ₱5,905.80
You save:  ₱1,771.92
¥10,806.63
List Price:  ¥15,438.71
You save:  ¥4,632.07
MX$1,423.91
List Price:  MX$2,034.25
You save:  MX$610.33
QR255.44
List Price:  QR364.93
You save:  QR109.49
P956.94
List Price:  P1,367.12
You save:  P410.17
KSh9,046.20
List Price:  KSh12,923.70
You save:  KSh3,877.50
E£3,475.04
List Price:  E£4,964.56
You save:  E£1,489.51
ብር8,733.50
List Price:  ብር12,476.97
You save:  ብር3,743.46
Kz63,853.79
List Price:  Kz91,223.61
You save:  Kz27,369.81
CLP$68,157.66
List Price:  CLP$97,372.26
You save:  CLP$29,214.60
CN¥506.84
List Price:  CN¥724.09
You save:  CN¥217.25
RD$4,224.90
List Price:  RD$6,035.83
You save:  RD$1,810.93
DA9,349.94
List Price:  DA13,357.64
You save:  DA4,007.69
FJ$158.89
List Price:  FJ$226.99
You save:  FJ$68.10
Q540.76
List Price:  Q772.55
You save:  Q231.78
GY$14,655.30
List Price:  GY$20,937.04
You save:  GY$6,281.74
ISK kr9,679.61
List Price:  ISK kr13,828.61
You save:  ISK kr4,149
DH699.78
List Price:  DH999.72
You save:  DH299.94
L1,274.52
List Price:  L1,820.82
You save:  L546.30
ден4,093.48
List Price:  ден5,848.08
You save:  ден1,754.59
MOP$561.60
List Price:  MOP$802.33
You save:  MOP$240.72
N$1,270.52
List Price:  N$1,815.10
You save:  N$544.58
C$2,577.69
List Price:  C$3,682.57
You save:  C$1,104.88
रु9,455.06
List Price:  रु13,507.81
You save:  रु4,052.75
S/265.82
List Price:  S/379.77
You save:  S/113.94
K281.99
List Price:  K402.86
You save:  K120.87
SAR262.75
List Price:  SAR375.38
You save:  SAR112.62
ZK1,938.58
List Price:  ZK2,769.51
You save:  ZK830.93
L331.04
List Price:  L472.94
You save:  L141.89
Kč1,685.42
List Price:  Kč2,407.85
You save:  Kč722.43
Ft27,354.11
List Price:  Ft39,078.98
You save:  Ft11,724.86
SEK kr771.99
List Price:  SEK kr1,102.90
You save:  SEK kr330.90
ARS$70,181.20
List Price:  ARS$100,263.16
You save:  ARS$30,081.95
Bs485.08
List Price:  Bs693
You save:  Bs207.92
COP$308,906.07
List Price:  COP$441,313.30
You save:  COP$132,407.23
₡35,572.52
List Price:  ₡50,820.06
You save:  ₡15,247.54
L1,770.13
List Price:  L2,528.86
You save:  L758.73
₲548,589.81
List Price:  ₲783,733.33
You save:  ₲235,143.51
$U2,994.50
List Price:  $U4,278.05
You save:  $U1,283.54
zł289.22
List Price:  zł413.19
You save:  zł123.97
Already have an account? Log In

Transcript

So welcome to lab eight and in lab eight, we will exclusively talk about blobs. So this is a one hour video. So in this will first try to understand what are the three different types of blobs that is blob blob, and blob page blobs. And in what scenarios to use use them. We will also talk about blob access levels you know, like container access levels and blob access levels and so on. We will see we will see the demonstration of all of these three types of blobs.

And while we are doing this demonstration, we will look into some very important properties like the threshold in bytes. The write and read size in bytes. We will talk about how to create chunks by using put block, how to create how to commit them by using put block list and then we'll also see that how we can use page blob, you know to do write pages to do read and seek. And finally what we will do is we will go and compare all these three blocks by using six criterias. So lab eight one hour exclusively blogs, blogs and blogs. So let us start.

So before we start with practicals, let us quickly go through some theory, and then the demos are damn easy to understand. So in the previous video, we talked about tables, you know where we said that tables stored in the name value pair, they have schema less and so on. But then we have scenarios you know where you'd like to store binary data, binary data means data like images, videos, PDFs, so in simple words, blob binary large objects. Now, binary data as compared to simple table data is huge in size. So it has to be treated a bit differently. So that's why as your has created a different architecture, when it comes to store blob data.

So the way the Blob Storage is organized is it is as follows. At the top we have a storage so the first thing is, we need to go and create a storage account. And inside the storage account, we need to create something called as containers and then inside the container, we go and use toward the blob data. Now, the blob data is huge in size, right? So when you talk about videos and images, they are huge in size. So if we can specify the nature of the operation to as your that as your can organize accordingly and make it more efficient.

So that's why blobs are of three types depending on the nature of your operation. The first one is a page blob, the second one is the blob blob and the last one is the append blob. So let us quickly understand that which type of blob to select, under what scenarios you will select them and how do you make the decision. So first thing it all depends on the type of operation for example, if the operation is when the append append means for example, let us say you are just doing a logging operation. You are just taking in print shots, and just logging it at the last. Then you can use append blobs.

If you want to do a random READ WRITE means if you want to just go and write randomly On the on the zR right then you can go and use a page blob. If you do not want to do a random read right, you do want to do a read write, but it is not random it is more sequential, then you can use the block blob. So basically depending on the type of operation, you can select these blobs. So if we only append append blobs, if you want to do a random reads right and search and so on page blobs, if you want to do it sequentially readwrite then block blobs. So let us first start with block blobs. In block blobs.

The binary data is divided into four four MB of size. So as the name says block, the binary that gets divided into four four MB of blocks. For example, now let us say you have a local file here which is of eight MB so it will divide it into four four MB that is two blocks and each one of the blocks will have an identification number. Now, these are Block blobs, when they are actually sent to your right, they can be sent in any sequence. For example, over here you can see this local file has been broken down into four pieces. So this is 1234.

But before the final commit, you know they can be in any sequence they can be 1423. But when the final commit happens, at that time, the same sequence is maintained, or else you know the integrity of the file will be destroyed. So you can see that when this final commit happens to the to the blob right to the blob, which is inside the container, it is again 1234. So in blob blobs, the data is divided into four MB size. And before the commit, you know, the data can be scattered in any sequence. But when the data gets finally committed to the blob, it's always in a proper sequence.

And these block blobs finally gets stored into the main blob and the main blob can have 50 2000 block blobs so that is a maximum. The The, the main block can have 50,000 blob blob. So in other words, every block blob is now a four MB, so four MB into 50,000 means the, the blob which is having the block blobs can be of maximum 195 gb, right. So just quickly recapping about block blobs first thing in blob blob. The data is chunked into four MB size. The data before the commit you know can be in any sequence but when they are finally committed, they are in a proper sequence.

The next thing is that the block blobs every block blob is a four MB size and the main blob block blob container can have 50,000 blobs block blobs that means 195 gb. So this was a theory part about block blobs. Now if you look at block blobs you know they divide the data into small chunks so they are very good for streaming purpose. So let us say you have one video and video you want to save it or the the images you want to store it or you want to stream it right because it puts them into chunks and these chunks are then again properly sequenced back right at the source block blobs or x are very good candidates you know when you have streaming data like videos or mp4 and so on. So now that we have understood blob blobs that is going quickly do a demo wherein we will go and we'll create a storage account we will create a container and then inside the container let's use C sharp language and create block blobs.

Because normally when you are when you will be uploading these blobs you will be using some kind of a language some kind of website or some kind of application right. So we will see now that how to go ahead and upload blob blobs using C sharp. So let us first go ahead and create an account inside account we will go and we'll create a container inside the container we will upload the block blobs right? Again, if you're not sure about where the mineral To create the blog just go ahead click on this create resource. There is a nice search button out here so I'll just say blob right? So there it is you can you can see that it says the storage account remember all this blob file table q whatever I have taken till now everything belongs to as your storage right so I'm going to click on the storage account I will say create all this is standard we have done this previously.

So I'll just say here, shift blob, okay, she was my name so I'll just push your blob for now. So that I do not collide with other names all these things we have discussed previously the resource manager the classic account kind and so on right over here you know I would say that please select the storage v to account because in view to account all the different types of blobs are supported, right. If you select must be storage v one, then probably page blob is not select is not is not supported, right for example if you storage Blob Storage, probably the page blob is not supported. So there are some scenarios out here you know, wherein, you know, some of the blob types are not supported. But in storage we do almost all the types of blobs are supported. So I'm selecting this, I'm taking standard, it's not thought it is cool.

Remember, if the data is accessed frequently then you need to actually go and select one of these. All this we have discussed previously, I will not go into this now, I will add all this to resource blob. So I'll create a resource here called as resource blob and let us do a create so the first thing is we need to create a storage account inside the storage account we will go and we'll create a container and then inside the container we will go and we will create the block blob, right. So there you can see my account is getting created. You can see this progress over here. So you can see See that our storage account has been created.

Please note the name issue blob. But this is a storage account, we can see I'm highlighting over its storage account. Inside the storage account, we need to go and create a container. Remember that hierarchy, there is a storage account, there are containers. And inside that containers, we push our blobs right? So I'm going to go and click on this plus here and we'll say my container.

I can see that there is a small drop down here saying public access level. This public access level defines that how much read access please note read access you want to give for the anonymous users anonymous users means users who do not provide any credentials who do not have any credentials. How much read access Do you want to give? So if you if you say private that means that no anonymous access is there for read and you need to log into your as your account to get access to this blobs and containers. If you say blobs were only the blobs are having access and not the container. So when did the blobs have a read access for when the blobs inside the containers, but not the containers outside.

And if you say containers, that means everything. So both the containers also he can browse, he can also browse the blobs inside the containers. So basically, this container public access level means at the container level you are giving access to the anonymous anonymous users to read it. And when you say a blob, only the particular blob only can go and read it right. So at this moment, I will just keep it at the blob level, right? And what is this thing can contain only lower cases that is strange, fine, I'll just say my container.

Okay. That was a strange validation, lower cases, okay. So I'll just say Okay, so now you can see, I have a storage account and I have a container and this container and we can access by using these access keys right So you can see that is the access key the connection string. So now let us go to our Google shisha program and let us try to push block blobs into this containers. So we have to block the storage account. Inside that we have my container.

And inside this mic container, we are now going to go and push log blobs by using C sharp program. So now here is a simple winform application I have created. So you can see here that this is a simple winform application. It's a simple winform form. And you can see that there are three buttons here. blob blob, and blob page blob.

So in each one of these buttons, I will be writing out sample code and demonstrations for each one of the types of blocks blobs. So now you can see the first thing what I've done is this you have done with tables as well. The first step is I went ahead and I use new gate and I downloaded or I added reference to microsoft.as your, so please go to nougat. And the first step is, you need to go and add references to microsoft.as your so you can see here, the Microsoft with Microsoft Windows as your storage is added right here once you're done with that, so the first thing is we need to go and connect. So step one, we will need to go and connect to the storage account to the storage account. So that's the first step.

The second step is and in order to connect to the storage account, we need this connection string this connection string, you will get it from the from the from the blob, so if you click on show blob, you can see that there is something called as access keys. So we'll copy this connection string and it will paste it here right. And as a best practice, normally, you will put this connection string in a config file and you should read it from a config file. But for now, I have hard coded it. Step two, you will get x you will get reference To do the blob client, right? So we'll get reference to the blob client and then step three from the blob client you will get reference to to the container right so from here you will get reference to the container.

So if you remember our blob name was Xu blob. So okay what happened to my mouse? Yeah, so our blob name was Xu blob. So you can see that's present in the connection string and inside that we had a container called as my container. So, first thing is this is standard for everything you are accessing tables, you are accessing cues you are accessing anything right you need to first get reference to the storage account. And then after that, we have got reference to the blob client.

And we have got reference to the container. Also why code you can create the container also means why code you can you can go and You can also create the container. Okay, I have at this moment I have done it via the portal. So the only thing which I want to create at this moment through the program is the blobs. Now, as we said, there are three types of blobs. So you can see the namespace the classes here.

So this is the cloud blob, so that it is cloud blob. So sorry, one is a cloud page blob, right? So that's a cloud page blob. Then we have the cloud. blob blob, blob blob, so there it is. And then we have the cloud append blog.

So all of these three classes are used, you know, to create the appropriate blobs. So this demo is about blob blob. So step one, we get access to the storage account, step two, using the blob client, we get access to the container. And then step three, we will use this container object and get access to the blob blob, right and then using this blob blob, we will try to upload some kind of binary content. For example, you can see, I'm trying to upload a PDF. Okay, so step four, use the container, use the container and get access to blob blob object.

And you can see I have created a blob blob called as ebook. And inside this blob blob, I'm uploading uploading a PDF. So it's a five step process. First get access to storage account, then get access to the container. And then inside the container, create a blob blob, and then using that blob blob, upload your binary content in that blob blob. So quickly reiterating.

At the top, we have Xu blob, this is the account, right? So the account storage name is Shu blob In that we have a container called as my container. And in that my container, we have a blob with the name ebook, which is of type blog blob, right? So we have at the top, the storage account name, which is shoe blob inside shoe blob, my container, and in that ebook, right? So let us go ahead and run this. Just quickly, this ebook PDF is approximately of 13 Mb size.

I'll tell you why I have told you the size right. So what I'm going to do is I'm going to go and run this project here. And while I'm running this project, I would like to go and run Fiddler tool. This Fiddler tool tells me that how is the data sent to the server? And why do I need this? I need this because remember I talked about that the block blob breaks your data into four MB pieces and then rearranges them and so on.

So we would like to see all that right. So at this moment, I'm not Doing anything I'm just uploading 13 Mb size and my expectation is when I upload this 13 Mb size it should break into four four MB right So, let us see what happens. So I have ran the Fiddler to let the back end and there it is running okay so it is running out here it is listening and let us run this and let us see what happens. So, there it is you can see that I've created three buttons I'll click on blob blob, and there it is it get access to the container then it gets access to the blob blob and I'm seeing Upload file so you can see now it is taking some time out here. But in my Fiddler tool, I can see the shoe blob right out here.

And when you see this shoe blob, you can see that it is actually uploading in one go. You can see there are other things out here but let us not look into that but you can see that there is only one request sent. And in that one request He has sent the complete 13 Mb In one go. So the question comes is what has happened? What? Where is that, for me breakup and so on.

Right? And you can see out here also the program has successfully executed and the file has been uploaded. So now the first question comes is that, why didn't he break this into Four? Four MB, why did he upload it in in one big 13 Mb slot, right? Because we did not instruct him. By default.

What count the cloud blob does is it says that okay, by default, I'll try to push this in one go. But if you want to push them into multiple pieces, right, then you need to go and specify me something called as single blob upload threshold in bytes. So at the blob client level, we have something called as default request options. So in this I'm saying that the single blob upload threshold value is one MB means what it says is that, if anything, crosses one MB If any of the data crosses one MB, start breaking them into pieces. At this moment, our file size is 13 Mb. So as soon as he says 13 Mb, he will first look into this threshold, it will say, Oh 13 Mb is greater than one MB, please start breaking them into four, four MB, right.

So let us see now what happens. So I'm going to go and clear all my sessions out here. And let us try to run this. Let us see. Again, I'm going to go and start my capturing it's capturing started. And let's run this.

So this time, I would I should see that it's broken down into four, four MB of size, right? So let's see now what's there, over here. So you can see now that's the first request, right? And if you look at this first request, can you see here it's four MB. Look at this data here. It's four MB okay.

So I'm going to go and stop capturing. It's done here. I'll stop capturing let us try to analyze this data I will stop capturing right here. So you can see here, there are four requests, five requests, 1234, and five. So the first the total file size is 13 Mb, so four MB plus four MB, eight MB, then this is again four MB. This is again, one MB, so you can see it for three is a 12.

And this is 13. I think the last one is a block list, you know, so it is not actually sending data out here. So you can see here, this ebook, PDF was sent in three block blobs, right? For mp4 mp4 Mb and one MB for blob blobs, right. So basically, we need to go and instruct we need to go and instruct the blob client that what is the threshold value, you know, after which, you know he should start Breaking up. So, if this threshold value is not provided, there is no way the blob blob will actually break into blocks it will try to upload it as one blob.

But now, we can have scenarios you know wherein we do not want four MB chunks we want must be eight MB chunks or must be we want to make one MB chunk for example, if let us say the end user says my internet is very slow, then probably rather than breaking the blocks into four MB, you'd like to break it into one one MB right. So for that there is one more property which we need to set and that is stream right size in bytes. So on the block blob, I can go and I can set your stream right size in bytes. Okay, so what this says is that this overrides that four MB chunk into one MB now so you can see this is one MB, remember it is in bytes, right? So that's why one zero to four 1024 it's one one MB now.

So, if you see over here, so, this stream right size in bytes it says that it sets the block size for writing to a blob blob right. So, what should be the size it is set here remember, do not confuse this with the threshold, the threshold is an upper limit which says that once this limit is crossed, then start creating the blocks and this suggests that what should be the individual block size again I repeat, this is a threshold at the block client at the top level at the container level which says that what is the threshold after which the blocks should start getting created right. And this says that what should be the size and it decides is not specified? And if you are exceeding the threshold and by default it is four MB right? So now let us see that does it go and create a four MB one one MB blocks, right?

So I'm going to go and run this person them out here. I'm going to go and click on this block blob, right and I'm going to go and run this. So let's go out here and let us see what happens. Now you can see here, look at this, look at this ebook, ebook ebook. You can see when I click on this the first one, you can see that this is one MB Can you see here? Look at the size here.

1049 Right one MB and remember, the whole ebook is 13 Mb. So this is 1-234-567-8910 1112 and approximately 30 right? So 13 Mb it has it has divided it has already done that for MB size into one MB. So in case if you think that your internet connection is if your end users internet internet connection is slow or fast, right, whatever it is, you can go and you can override this and you can make the upload more efficient. Also like You can see I've said upload from file here you can also say upload form file a sync. You can also say upload from stream so that our upload from byte array right.

So, we have three four flavors here like upload from byte upload from file upload from stream, and each one of them also has upload from file a sync or upload from stream esic. So, you can also go and use the a synchronous method, if you use a synchronous method, it will not block this call, it will just go to the next call. And still as synchronously at the background, the upload will happen. But now you need further more customization. For example, if you see over here, the maximum threshold that you can have is one MB. So in case if you want to create a chunk of 512 kB or 1010 kB right, you cannot do it it has to be minimum one MB so this is the first drawback here.

The second drawback here is which is more important than the size is that over here the chunks are created automatically and uploaded. But what if I To do like this, I want to create a block of blob blob of let's say 10 chunks. And I want to go go like, you know, first I do five chunks, then I pause, then I resume. So, I want to give a resume pause continue kind of fun thing. So for that, you know, I need to go and create the blocks manually, right? upload the manually so that I can plug in the feature of pause and resume right.

So for that, you know, on the blob level, there are two methods which is given one is called input block. In this put block, you can see that there is a block ID and there is a block data. So you can give your own ID whatever you wish, and you can give your own data what you want. And once you are done with this, you can then go ahead and use the put block list. So you can use put block two, you can use this put block to add the blocks. And then you can use the put block list in order to go and commit the list.

So you can see there's a put block list and this block list you actually have the you have to give the list here. saying that what is the sequence by which you want to commit so put block to add individual block items and then finally put block list to make the final commit with the sequence for you want so we need to use both of these methods and you can go and you can create your own manual blocks right you can customize it you can pause it, you can resume it and so on. So what I've done is I've created one ready made code out here because this will need a lot of coding so what I've done is I've created this upload method and in this upload method I've written all the code what is needed for creating the block list and and so on right.

So we will be using the same file the PDF file and let us see that how we can do the the the creation of the blocks and uploading it manually. So here this first step is in a where we give the put block name of the block name right the blob blob names or blob name will be ebook, right? What is the blob I'm going to create let us say one MB. Then the next thing is I am giving him the file name and I am reading the file length. So here I am reading the file length, right. So the complete file length divided or the file size divided by the block size, what I have decided is the block count.

So, for example, if my file length is 13 Mb, and I'm seeing Okay, like I want to get one MB, right, so 13 by one plus so that 13 blocks will be created, right? Then I'm creating a string here. And in this string, I will go and I will create my block IDs. Now this is a very important part. If you remember in the first thing, you know, when we were using this automatic method, I cannot give my own block IDs, he actually creates his own block ID. But now that we have taken up the whole customization process, I can go and I can create my own block IDs over here.

So here I have a list of strings of block IDs. So what I do is I start reading the file so you can see that I'm I'm reading the file Until the file size, and I create those chunks here. So I'm seeing Okay, whatever is a byte life left's there, I'm just creating the block size out here. And then every time a block is created, I'm adding it to the block ID. So you can see I'm adding it to the block IDs to the block ID collection, I can see this block ID out here is using the base 64. conversion. Now remember, the block ID cannot be like simple string, please note.

So here you can see my name is block ID, and whatever is a number. So whatever is account at that moment, so that is my block. So like, I can have block one block ID one block ID to write, but I need to go and convert this to base 64. So you can see here, I'm converting it to base 64. And then I'm adding it to my collection. Okay.

And then I'm using the put block. So whatever is the ID, I'm using that ID and I am reading the File Stream, and I'm also adding the hash and hash just is for the zero to say that Please verify you know if whatever data I'm sending is properly matched or not, or it is the integrity is maintained or not. So in other words, here I use the put block method and add my blocks. And then finally, when everything is done, I'm saying put block list. So you can see this while loop out here, it's actually going and looping until the whole file is completing right completed reading right chunk by chunk. And every time I'm reading a chunk, I'm using the put block.

And once I'm done, I can go and now say put block list. Right. So again, this one out here will have the same kind of output, this will take out 13 Mb file size, and divide into 13 blocks. So So let us see how it looks like. So what I'm gonna do is I'm going to go and comment all this thing out here. So this is all commented, not needed, right?

This Everything is there internally. So over here, I'm going to go and use that upload method what I've created this code is that inside this Visual Studio, so I'm going to go and upload The code. So wherever the video is running at the left hand side, you can get this code, right. So I'm going to go and pass in the container, right? And I'm going to go and pass in the file name. So the file name is the same ebook.

So what I'm going to do is I'm going to go and pass in the same file name here ebook. And let us see that if he goes ahead and if you create 13 Mb size or 1013 blocks, right, I would go and clear all of this, I would go and clear all of this out here. A lot of things out there and we'll clear it. Okay, so let us see how it looks like. So I'm going to go and run this assuming that everything will go right. Okay.

So I'll say block blob, right. And I run it. So once I run it, let us quickly go to the Fiddler out here and you can see now that it is you can see our blocks getting created. And these blocks are actually getting created manually. Please note, I'm not using the Upload file at this moment. I'm using the put block list.

So over here 13 junk should get created 1-234-567-8910 1112 13 and 14. I don't know there are 14 chunks. So that is right actually 14 chunks because my file size is 13 point some x MB, please note, my file size is not exactly 1313 Mb it is 13.6 Mb. So there are 14 blocks. And that is right. And if you see in my code out here, I have said here, that basically it is if you look at the block, the block size, so whatever divided plus one, so that's why for the last remaining bytes, you know, we need that one extra block size.

But you can see here that yes, you know our file has been uploaded, and it has been uploaded in 14 chunks, right. So there are two ways to do it. One is you can use the ready made method out here upload from file upload from byte array upload from stream and so on, or the other ways that you can use Put block lists and you can use put block to do it manually they control so that you can do pause, resume and so on. Now, if you want to also go and loop through a block block for example, I want to go and look through a blog blog. And I want to know that how many blog blobs are there in this blob here. So, what I can do is I can go ahead here and get all the block lists by using download block list right.

And if you look at this download block list it also has an input parameter here saying that what can be the block filter for example, do you want to see all only committed logs Do you want to see all blocks or do you want to see uncommitted blocks What do you mean by uncommitted blocks, blobs, uncommitted blobs means for example, you had called the put, write you call the put method. If you remember we had done this previously, you called the put method but for some reason you did not call put blocklist so there was no commit happened. But it is there on the As your account right, so if you want to go and query such kind of uncommitted list you can also go and query right. So at this moment, I'm just going in querying all the block list, it's either committed or either uncommitted and remember we have 14 block blobs in this ebook right.

So, I hope that we we get to see all the 14 So, let us go and see if that happens. So, download block list, you know helps you to go and query that how many blocks are there so you can see this is that it is the first block, right? This is the second block right and you can see the length here in the length it is one MB look at that 1048576 it is in bytes it is one MB second block, third block, four block, fifth block. So there it is, you can see all the 10 blocks and creating. So if you want to go and query that how many block lists are there in a particular block blob, you need to use this download block list and query it. And also if you want to go and query the committed uncommitted, you can pass it back meter and also get the committed separately and uncommitted separately.

And if you want to go and download this complete Block Block Block Block, right so what you can do is you can say block block dot download file a sync or you can also use download file here so blob blob dot download file. But you can also use download file, download to file so you can see it in the C drive. Let's say I'm going to say my PDF right and I want to download that remember the 13 Mb is the size let us see if it works or not. And at this moment, let me just also quickly go and clear everything out here. Let us see that is it downloading in chunks or what is happening right so I'm going to go and remove all sessions. I'll start my capturing right here and let us see that.

It is downloading in chunks. I did not expect him to download in chunks because I'll tell you why so, so that it is download file, let us quickly check is it downloading in chunks. So you can see that he is not downloading in chunks, you can see this minus one here. And I can see that he's trying to get the full block out here, right. So if we just quickly go back again, you can see that 13 Mb. So in my C drive, there is a my PDF.

So if I go to my C drive here, there is a my PDF and if I go to this my PDF, and if I see the size out here that is 13 Mb so you can see like, he just took all the block blobs, all that all that 13 or 14 block blobs combined it and send it out, right. But what if I want to go and download it in chunks, right? What if I want to go and download it in chunks the way we have uploaded in chunks. So then what you can do is you need to go and give one more property here. The way we have right bytes in the same way. We have here.

Read bytes, so I can see a stream, read read bytes read minimum read size in bytes. So you can see our minimum read size in bytes. And when you use minimum read size in bytes, right if you if you really want this minimum read size in bytes to be effective, then you have to go and use open read a sink, right you cannot go and do the normal what I've done previously. So you can see I'm saying open a blob stream and say open read a sink and I'm going to use the await Remember, a sink and await in case you're new to this keyword of a sink and await it is specifically for as synchronous calls. So if you want to know more about async and await please go and see this section you know where I've explained what is a sink and await but if you haven't await here I need to go and define a sink over here so I'm gonna go and define a sink.

Again like in this video, I will not explain what is a sink await you can please go and see this video which I'm flashing on the screen right away. So now if you see it Now in this case I'm expecting you know that this PDF should be downloaded in one one chunks means one one MB chunks right? So I'm going to go and remove all the sessions from here. And let us see that this works or not. So I'm going to go and click here and say upload. And I hope that I get to see all I get to see chunk download so you can see it now look at it.

Can you see that he is downloading in chunks of one MB look at this size here 1111 mB 14 chunks whatever was the was the block blob by which we uploaded, those are downloaded. And it is my new PDF. So if you go and see this my new PDF there it is. And that size is also of 13 Mb. So you have the facility you know to go and upload in block blobs, you have the facility of downloading in block blobs. So that's why you know I said in the in the first part of the video when it comes to streaming when it comes to streaming binary blob data.

It is Always blocked blocked. Okay, so that was all about block block. Now the next block or the next block we will discuss about is append block. So let's talk about open blob. Again you know when you are actually working with blobs Do not forget your friend as your Storage Explorer. in one of the previous videos we have explained the Storage Explorer out here so we can see at the top we have the storage account shoe blob, right inside the shoe blob, we have the blob containers.

So there was my container. And inside this my container we had our ebook blob, right so you can see a blob, right? So what I'm gonna do now is I'm going to go and you can see by mistake I've created other blogs as well here. So I'm gonna go and delete all of this for now I'm gonna go and delete all of this. Now, remember that once you create a blob with a type, block, blob or append blob, you cannot add other blobs to it. blob types to it.

For example, this ebook A type blob blob you can see here this type out here it is a block blob right? Now in this block blob, I cannot go in an append block, or I cannot go an Add Page block, right. So I'm gonna go and delete this. So let us go ahead and create an append blog programmatically. Remember, always keep deleting, always keep deleting, this is all costing you your credit card is attached right to the has your account. So I just ensure that I keep deleting it.

So for append block, if you remember in my screen, I had created three buttons. So all the code of block blob is right out here. And there is another button out here append blob, you know where I'm writing the app and blob code. So in my G drive, I have Hello one and hello to dot txt. You can put images you can put PDFs right. So again, the steps are almost same, you have to create the object of storage account, you have to get access to the container and inside the container.

I'm creating a my blob, append, write and in that I'm appending to files Hello one dot txt and hello To dot txt, so, let us go and run this and let us see that how it looks like. So, let us run this let us see what happens if I click on this if I go here if I go here oh it is throwing an exception saying that this is not found. So, basically this append blob right does not really exist, right. So, what we have to do is we need to write a simple two lines of code here, we can see that if append blob does not exist, so, over here I can say.if it does not exist, then create one So, create or replace, right and we just create this exists as a method. Right. So, hope that this works again I'm gonna go and put a debug point out here.

Always remember if you get errors, it is very good. Because then that way you learn more so you can see if it does not exist, it should Go inside this, I will now create the blob. So if I go to my Storage Explorer, I should see the blob out here. So if I do a refresh, I should see a blob here, Refresh. Refresh, you can see, that is my blob app is an append blob. Very nice.

Inside this I'm gonna go and append two files. Hello, one dot txt and hello to dot txt. So now in this blob out here in Azure Explorer, we have two files, you know, which have been uploaded, right? So if I double click on this, you know, probably we can see those two files, you know, so I'm gonna go and double click on it. And once I double click on it, he says that I don't know what kind of data is this because for him, it is just binary, right? So it's showing a potentially unsafe thing.

So we have to specify him in which application I can open so mostly I can open that in Notepad. And you can see that Hello, one and hello to both have been downloaded, right? Again, if you want to go and download an append blob Same thing what we did previously, you have to go and say here append blob dot download to file and you have to give the name. Remember the same syntax, what we used for blob blob, right? The only thing that happened blob is that you cannot go and update it. For example, in the block blob, I can go and update a particular blob if I want, but in this case, I can just append it, okay, or I can just purge it if I want finally.

So that was a simple syntax of append blob. Now the next thing what we will do is, we will do the page blob, right? So we have done block blobs we have done append blobs, and now the last one is the page blob. Now again, four page blob or blob blob, or append blob, right, the first thing is you need to get access to the storage account. You need to get access to the client you need to get access then to the container. So all these steps are same.

The next thing is we'll go ahead and we will get Access to the cout cloud page blob so you can see here i'm i'm telling the container please get me play, create a page blob called as page blob 123. And you can see I'm seeing your that if this page blob does not exist, you know, please create it with three into 512. That means, take 512 what he called KB and then create three chunks of it right? Now why am I saying three into 512 kB if you remember, we had said that in page blob, we write 512 kB chunks. Why? Because we want to do random read and write easily.

So you can start from zero bytes and you can say 512 kB is here and then again, start from zero bytes and say one zero to four. So for example, when I say three into 512 kB, I'm going to say I'm going to go and create a full page of 1536 kB which is divided into three parts of 12512 kB each. So I can go and write on page one from zero to 512 kb. The next page is from 512 kB 210 to four, and then after that 102421536, right? So basically that's what I'm doing here, I'm creating three empty pages where where I'll be writing data. Now let us say that we if you remember in the in the append, blob, we were using this Hello one and hello two, right.

So let us say, I want to go and write this Hello, one a, you know from zero to 512 kB, and this hello to from 512 kB to one zero to four. Right? And let's see, after that I want to just leave the other pages empty. So for that the first thing is we go and we'll create a byte array. So you can see I'm creating a byte array of 512 kB, right and this byte array. You can see that I'm seeing over here this byte array.

We'll be reading the Hello dot txt file. And I'm writing From zero to 512 bytes, right? So that is the first thing, the next after 512. You can see I can see here now. Now this is the next thing which I'm doing here. So here you can see I'm writing the page from whatever it is this memory stream.

I'm writing it from the zero offset look at this syntax here PB dot right pages, this PB is nothing but it is this page blob, right? So I'm reading this file into a 502 kB chunk, and then I'm writing from the zero offset from the zero offset in the 502 kb. I'm writing Hello one dot txt and you can see the next line from the 512 offset means from from this section, I'm writing the next byte. So basically, the Hello one dot txt is present from the zero opposite to 512. And hello two dot txt is present from 512 kB To one zero to four, right and the other things are all empty. So there it is.

So if I now go and run this, let us see what happens. So let us go ahead and run this. So this will actually create page one in the first offset, first thing, what it will do is it will actually create a page of a page of 1536 kB, right. And with 512 kB chunks so you can see this is the first syntax here, I'm creating three chunks of 512 kb. And then I am first reading the Hello one dot txt and I'm writing it. Then I'm reading hello to dot txt and then I'm writing on the second chunk.

Right. So now at this moment, if you see I have three pages, which is written in this page blob, right HelloWorld dot txt on the first page, hello dot txt on the second page and the last page is empty, okay. So if you want to go and read If you want to just go and read that, how many how much of the page ranges and so on, or we can really do is we can go ahead here. And we can read it by using this get ranges so you can see it, what I'm going to do is I'm going to uncomment this. So now in this I should get at least three ranges, right? So let us see.

So I'm going to go and just see that how many range of pages have been created. Right? So I'll go out here I'll see the container and there it is. So we have the first range you know, which starts from the offset zero to the end offset, one zero to three. Oh, this is good. Actually, you can see here at this moment, it is just saying that in this page, how many ranges have been written.

This does not tell you the number of pages okay. So we can see that at this moment. It is not telling that we have three pages in this book. It is just telling you what is the end offset. And if you want to go and get the specific chunk, then you need to specify from where to where, from, from which position to see what right. For example, if I want to go and read data, then I will do something like this.

I will open up the page, blob stream. And then I can say, okay, read. Now this file from this range from zero to five one to look at this code here, I'm going to go and copy this code. So if you see here, so this Ranger just talks about, you know, the, the start and the end, he does not talk about the number of 512 kb. Okay, if you want to get the number of five and two KB, you have to just divide it I'm sorry, I said that. It tells you the number of ranges but not exactly the number of ranges.

It just tells you that what is the start and the end, okay. Now if I want to read let's say file, one dot txt, or sorry, hello, one dot txt, right. So what I can do here is I can go here and see that okay, create a blob stream Open this cloud plays blog to read and then read it into this data and let's say we write it into some file, we're saying new one dot txt, okay. So now if you see here so in order to read you have to say, which is the start opposite of the byte and which is the end offset of the byte, and then in this byte array, that data will be read. Okay, so I'll go and run this. So there it is.

So first thing it, it checks you know, if this page blob has been created, which has already been created, I can see I'm reading this range from zero to 512 kB and I'm writing it to new one dot txt. So now if I go and see this new one dot txt, which is in my D drive, I should see that Hello one year and if I read from the other range, so for example, if I try to read from let us say From 512 okay if I want to read now the next range if I want to now read the next range then I need to first go and move this cursor you can see here I'm seeing your blob stream please go ahead and move to the five one to start so that is the next chunk right and then from here now go and read 512 kB now this will actually go and read the next Hello two dot txt okay so you can you can see here without the seek you know the current cursor will be on the zero offset.

So when I say that okay, seek Now move ahead to the 512 offset, it actually goes to the next offset right so yeah, this actually will now start reading from 512 offset. Okay. So, so in this case now I should get hello to dot txt so page blob again Create, and it writes to the new dot txt. So if I go and see the new dot txt, I should see hello to. So you can see like, if you want to go and read data, you have to first go and use the seek and move to that offset. And then you can go and read data.

And also if you want, you can write a chunk on that seek as well. So it will it can also go and replace the data. So remember, page blob is to read and write quickly. Right? blob blob is meant for streaming purposes, you want to break things into chunk and then you know again, rearrange it while append blob is to just add it to the last and append blob in app and blob we cannot do an update. So that was all about page blobs.

Now let us do a quick comparison between all the three blobs here. So you can see I have used some six criterias you know on which we can put down the comparison So first one is when you talk about writing and updating. So in case of writing, you know, when you're writing a blog blog, you can use a block ID and then specify the chunk of the data. While in case of page blob, you have to specify the start and the length. And while in case of an append blob, it is always at the last so it just gets appended at the last. When you talk about updating, you can again update the block blob and page blob by using the block ID and offset length.

But in case of an append blob, there is no update as such. The scenario for which you know you'd be using block blobs and page blobs are as follows. So first thing when you talk about streaming, if you if you remember, in blog blogs, we can break them into small chunks, right? And then again, reorganize them. So that's why in a blog blobs are very good for streaming purpose. While in case of page block, it is good for read and write.

The chunk is small, but you can quickly go and read by using the offsets you can quickly go and update and write by using the offsets in case of append blob it is for logging purpose for example, you have no screenshots which are coming in every second or something. So you can just keep appending at the last. When it comes to size the individual size of every of these chunks in blob blob is four MB in append global so it is four MB in case of page blob, it is 502 kB and the total blob size that is a spelling mistake the total blob size in block and append is 195 gb because you can put you can fit 50,000 block chunks into one big blob. While in case of page you know there is no limit as such. And when it comes to commit in case of blob blob, you you have a two phase commit.

So in other words, you know you first break them into chunk and then you can then say commit all the chunks right? While in case of page blob and append right, the commit is instant it is instantaneous. In other words, you take that page blob and you just say commit it is done. Okay. So these are some of the criteria you can remember So when you come across any scenario, right you say that okay, which blob I should use Now, first ask yourself question that how these six six criterias are determined. So if you can, if you can look at the nature of these six criterias then you can choose you know which of the blob is good for you.

So, that brings us to the end of this blog video. So, in this video, you know, we were trying to understand the three different kinds of blobs. So this source code, I'm uploading it right on the top of the video, so you can go to the source code, and you can see all the samples which I've explained. So in the blob blob button, you can see the block blob samples in the append block, you can see the append blob samples and in the page blob, you can see the page blob samples, right. So thank you very much. And the next video after this would be on queue, right.

So if you remember in the storage types, we had blobs which we have completed, we had tables you know that also we have completed and then our The next thing is queues. Thank you very much.

Sign Up

Share

Share with friends, get 20% off
Invite your friends to LearnDesk learning marketplace. For each purchase they make, you get 20% off (upto $10) on your next purchase.