Due to loss of old server, source is not available. Links are being stripped until code is put back up online.

The Omniscientist.Cloud.SQLServices library has been updated to include BLOB functionality. It has also undergone a significant structural overhaul. I still consider it to be in its infancy, but I think it has started to become useful.

A sample application has been developed in order to showcase the new functionality, by the name of ‘Cloud Photos’. It allows the user to upload and view images stored in the cloud; specifically, in a dedicated SQL Data Services container.

Download the ‘Cloud Photos’ sample application binary here.

View the ‘Cloud Photos’ source code here.

Read on for a brief overview of BLOB’s w/ Azure SQL Data Services, and how to manipulate them with my library.

BLOB’s: Overview

The concept of what a BLOB is remains the same with Azure SQL Data Services; they are what they are in any other storage environment: binary data.

BLOB’s, like so many other levels in SQL Data Services’ containment hierarchy, is a type of Flexible Entity, similar to others that we have already encountered (Authorities, Containers, and Entities).

You may remember a previous picture I (hopefully) painted in your minds: SQL Data Services is made of a containment hierarchy model, the Authority level being at the bottom (the base, foundation, etc.), Containers being one level on top of that, and then Entities being at the top, containing the actual data. BLOB’s are neither above nor below Entities in the containment hierarchy model, rather, they exist alongside Entities.

BLOB’s have, in common with Authorities, Containers, and Entities, the Id, Kind, and Version properties. There are no surprises with the Id property, but the Kind property, although writable, will always default to “Entity”. Azure documentation seems to hint that this will not always be so, but until then, it is important to remember this when designing the other levels of your dedicated containment hierarchy.

Although there are other ways to differentiate between Entities and BLOB’s, I feel the best way is to never label your Entities’ Kind as “Entity”. The Entity’s Kind property is crucial in that it allows you to distinguish between different types of Entities, in case there are multiple types in a single container. If you leave an Entity’s Kind at “Entity”, you are going to find yourself in a rut, as there is no other reliable way to differentiate between BLOB’s and Entities in a single container.

Unlike the other data model levels, BLOB’s host a unique property: Content. This property in itself, contains three attributes:

  • content-disposition
  • content-type
  • content-length

Remember, BLOB’s allow the storage of binary data; naturally, the above attributes relate to this binary data. The content-type attribute refers to the MIME content type of the binary data. The content-disposition refers to the content disposition extension to MIME. The content-length attribute is self-explanatory.

Notice that nothing contained in the Content property has anything to do with the raw content itself. It may not be immediately clear where, then, when dealing with a BLOB, how one goes about retrieving or storing the binary data. I’ll come back to this later.

Also unlike other Flexible Entities, BLOB has nothing “flexible” about it; it lacks the Properties property (which I call the Body property in my library), which means a BLOB can have binary data and only binary data (in addition to the Id and Version) associated with it.

Good to know, but, how do we use them?

Excellent question! You may remember that you can use two different protocols when working with SDS: SOAP and REST. Indeed, my library contains both protocols available as individual clients. I can consider that fact useful as Microsoft does not provide any sort of client for the REST protocol, just specifications.

The SOAP interface, interestingly enough, is not supported when dealing with BLOB’s. The only way to interact with them, is through the REST protocol. So if you’ve been relying on Microsoft’s SOAP client for basic SDS operations, and are ready to take the next step, you are out of luck (well, until you happened to stumble on this site, that is).

In order to upload a BLOB to the cloud, you must obtain a stream to the data you wish to include, and then make an HTTP POST request to the appropriate container. You must specify the Content-Length and Content-Type properties, as well as the Content-Disposition header when posting a BLOB. Obviously, the Content-Length is not optional, but the Content-Disposition is. Not specifying the Content-Type is a bad idea.

Another header you must specify in the request is a header named the “Slug”. The “Slug” header, is actually the BLOB’s Id. Slug is a cooler word than Id, so I’m down with that terminology.

Finally, you must also specify the Authorization header with the correctly crafted credentials. I’m not going to go into the details of that, as you can see it in action by looking at the Omniscientist.Cloud.SQLServices library code.

Once you have the request crafted, the fun soon begins. After writing the first chunk of our binary data to the request stream, we need to establish an asynchronous pattern in order to allow the uploading of the BLOB to occur on one thread, while at the same time downloading the response on another. In order to retrieve the data, you simple make a GET request on the BLOB’s URI, which will be your Container’s URI concatenated with the BLOB’s Id.

There’s more to say here, but I’ll leave it to you to find the details on MSDN or by looking my library (or, hell, ask a question). Let’s look at how my library, using CloudPhotos as an example, makes life easier for you when you need to deal with SDS and BLOBs.

Using the Omniscientist.Cloud.SQLServices Library

If you look at the CloudPhoto source, you will notice that all the functions that deal with the cloud (i.e. calls to my library) reside in the CloudWorker.cs code file. You will also notice, that this file contains almost no code. This is a good thing, and it indicates that you can avoid dealing with all the required crap when working with SDS on the cloud, by using the Omniscientist.Cloud.SQLServices library.

If you can recall, I pointed out earlier that BLOB’s are not flexible at all, and can only contain binary data (+ its Id). That’s unfortunate, as what is the point of having a bunch of raw data if you can’t associate other, more human recognizable, pieces of data such as names, dates, or descriptions with them.

How do we get around this problem? Well, when you create a BLOB using my library, what actually happens is the creation of a foreign key relationship between a newly created BLOB and Entity pair. The BLOB Id gets stored in the Entity as “BlobId”, along with any other properties. Don’t worry, this doesn’t mean you have to deal with multiple data instances at once, this is all done transparently, and everything is returned as the desired type or Blob object.

If you’re still with me, good. Bearing that in mind, here is how I suggest using the library in order to manage your BLOBS:

  1. Define a class that contains your BLOB-Entity pair’s flexible properties. For example, the CloudImage class which contains the “Name”, “ImageRaw”, and “Image” properties.
  2. Mark the property which will contain the binary data with the [BlobData] attribute.
  3. If you want the library to ignore a property, mark that property with the [BlobData(true)] attribute; the parameter being the “ignore” parameter, as in you want that property to be ignored.

Let me go over what’s happening here. Looking at the CloudImage class, you’ll see that the ImageRaw is marked with the BlobData attribute, the Image property is set to be ignored, and the Name property is left alone (meaning it will be included as flexible property).

This results in a Blob-Entity pair consisting of:

  • Binary BLOB data (created from ImageRaw)
  • Id
  • Body (our flexible property collection consisting of the Name and ImageRaw properties)

Notice what we just did here: They said BLOB’s couldn’t be flexible, well, we just made them flexible, and all of this is made available in a Omniscientist.Cloud.SQLServices.ServicesModel.Blob object.

The property marked as the binary BLOB data source property (ImageRaw in our case), must either be: 1) in byte[] format or 2) serializable. The reason for this is, because I need to create a Stream using the BLOB binary data, unless the object is serializable, it’s required that it is available as a byte array.

Why would we want to ignore a property, namely the Image property (which is an ImageSource instance)? ImageSource, as you know, is not serializable, so I need it in byte[] format. It would be mighty inconvenient, however, if I had to convert that byte[] to an ImageSource every time I had to look at my image collection, so once we have the byte[] we convert it to its original type.

We don’t want this ImageSource instance near our cloud storage services; I don’t feel I need to explain to you what sort of complications this would cause.

I hope you understand why ignoring some properties is important.

The CloudPhotos application retrieves a bunch of images from a dedicated container that exists in the cloud, and allows you to upload a new image to that very container. This is all done using the following code snippets (the Images variable here points to a list of CloudImages):

Retrieving Blobs

foreach(CloudImage image in _client.SearchBlobs(null))
    Images.Add(image);

Hmm…pretty…simple. The null being passed, btw, is in place of a query, and simple returns every entity in the container.

For the uploading blob example, note that the unboxedImage is the new CloudImage instance we intend to upload.

Uploading a New Blob

Blob newBlob = Blob.FromTypeDefinition(unboxedImage);

_client.UploadBlob(newBlob);

I’ll provide more in-depth (ala Sandcastle) documentation in the future.

Enjoy.

Matt Weber

I'm the founder of Bad Echo LLC, which offers consulting services to clients who need an expert in C#, WPF, Outlook, and other advanced .NET related areas. I enjoy well-designed code, independent thought, and the application of rationality in general. You can reach me at matt@badecho.com.

 Leave a Reply

(required)

(required)

You may use these HTML tags and attributes: <a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

 
   
© 2012-2013 Matt Weber. All Rights Reserved. Terms of Use.