« Red-Gate’s SQL Compare – now with SQL Azure support! | Main | Running multiple apps on the iPad at the same time (Multitasking) – now! »

How to use the new Windows Azure CloudDrive feature – two screencasts available

Today I gave a presentation to my co-workers Sung P. and Lanh N. on how to use the new Windows Azure CloudDrive feature.  I thought it would be a good idea to share what I found on the Azure CloudDrive with the .NET development community.

09-myvhdwebrole-screenshot
My sample Windows Azure Web Role mounts a VHD
from Azure Storage and reads text files from this ‘CloudDrive’

In the screenscasts, I show how you can create a VHD file, format it and upload it to Azure Storage.  I then show how a Windows Azure Web Role can retrieve the VHD file from Azure Storage, cache it locally and mount it as a NTFS drive.

I created two separate screencasts which you can access below:

  Windows Azure CloudDrive -
  Creating a VHD and Uploading it to Azure Storage (Part 1 of 2, 30 minutes)
  http://screencasts.ehuna.org/2010/03/windows_azure_clouddrive_creat.html

  Creating a Web Role that mounts the CloudDrive
  from Azure Storage and reads text files (Part 2 of 2, 22 minutes)
  http://screencasts.ehuna.org/2010/03/windows_azure_clouddrive_creat_1.html

Each article contains the screencast and notes; you can also download the source code of the Visual Studio 2010 RC sample projects used in the videos:

  http://blog.ehuna.org/files/Visual-Studio-Solution-AzureSpeed1.zip (134 KB)

Below I have screenshots, links, and additional notes on Windows Azure CloudDrive that I used during the presentation.  Good Times!


Windows Azure CloudDrive
VHD – NTFS Drives in the cloud

Summary

These are general points on the Windows Azure ‘CloudDrive’ feature.  These are in no particular order – but they are important in understanding in what scenarios a CloudDrive can be useful and what potential problems you may run into.

  • Windows Azure SDK 1.1 (Feb 2010) released a new feature: CloudDrive.
  • When running in the fabric an Azure web or worker role can mount a VHD file - which is then available as a hard drive.
    • The VHD is stored as a Page Blob in Azure Storage
    • The minimum size of a VHD is 16 MB.
    • The maximum size of a VHD is 1 TB (which is the maximum size of an Azure Storage Page Blob).
    • You are only charged for the pages that are stored - for instance, you can allocate a 1TB Page Blob and if you are only storing 1GB of pages in the blob, then you are only charged for 1GB of storage
    • The VHD must be formatted as NTFS (FAT is not supported, I have a request here).
    • For smaller drives, NTFS uses up a lot of space
      • 20 MB VHD: only 7.9 MB of free space after initial formatting.
      • 25 MB VHD: only 12.8 MB of free space after initial formatting.
      • For larger drives the additional security headers of a NTFS drive do not come into play.
  • Only one web or worker role can have read/write access to a specific VHD at any time.
    • You need to get a "lease" on page blob for this to work - I have not done this yet (no demo code for this).
  • For read-only access, you create a "snapshot" of an Azure Storage Page Blob (VHD), which you can then mount.
    • Snapshots are small in size: roughly 512 bytes, they are only a pointer to the actual page blob containing the VHD.
    • Remember to delete the snapshot and un-mount the drive on role shutdown.
  • The VHDs can be cached locally in your web role's local storage.
    • The local "on-disk" cache maximum size depends on the VM size you chose for your web/worker role.
  • Once you mount a CloudDrive you get a drive letter and you can use System.IO classes to read/write to files.
  • Important: mounting a CloudDrive only works in the Azure fabric, local development fabric or Staging/Production cloud.
    • You cannot mount the drive in an ASP.NET site running in IIS, a console app or Windows Forms app.
    • You can only mount a CloudDrive from the same Azure storage you're running in
      • If you are running in the local development fabric, you can only mount a VHD from local storage.
      • If you are running in the Staging/Production cloud, you can only mount a VHD from Azure Storage.
    • I have a trick I came up with if running in IIS for fast development: add the files you need to source control; create a function that returns the folder containing the files you need to access and at runtime do this:
If Not (RoleEnvironment.IsAvailable) Then
' So we don't get an error when running in IIS
sFolder = System.AppDomain.CurrentDomain.BaseDirectory()
Return sFolder
End If


  • Good overview document: "Windows Azure Drive - February 2010v2.docx" - see the "Usage Patterns and Best Practices".

Example: How to read text files
from a VHD in an Azure Web Role

In this sample projects I cover in the screencasts, we create a VHD, copy in some text files, and deploy the VHD to Azure Storage; then in a web role we mount the VHD as a CloudDrive and read the text files.

  • Create a VHD

08-creatingvhd-disk-management




  In Windows 7 and Windows 2008 Server R2 
  you can create and mount VHDs natively
 


    • Can be done through code
    • Can be done manually in Windows 7 and Windows Server 2008 R2 through Disk Management.
    • We use Computer Management > Disk Management > Create VHD to create a 20 MB VHD.


03-creating-vhd-files


A mounted VHD in Windows 7 looks like a regular hard drive 


  • Upload the VHD to Azure Storage (Page Blob)

04-myvhddrive-code-sample


Sample code to get a reference to a Page Blob in Azure Storage


    • Can be done through code
    • See sample project 'MyVHDDrive' - references DLLs from C:\Program Files\Windows Azure SDK\v1.1\ref
    • We might also be able to upload the VHD to Azure Storage through the latest version of Cloud Storage Studio or other GUI clients (download worked).
    • In development the VHD storage is a simulation, so the same method cannot be used. Instead there's a trick to create the VHD and copy files manually.


01-localstorage-vhd-simulation-nonsnapshot-screenshot

      • The directory is normally: C:\Users\[USERNAME]\AppData\Local\dftmp\wadd\devstoreaccount1\[CONTAINER]\[DRIVE.VHD]\[FOLDER]
      • If you ran into the path max length issue, and you have defined _CSRUN_STATE_DIRECTORY, let’s say as “C:\t” then: C:\t\wadd\devstoreaccount1\[CONTAINER]\[DRIVE.VHD]\[FOLDER]


  • A Web/Worker Role mounts the VHD

    • In your project, add references to the Windows Azure SDK DLLs, including the new Microsoft.WindowsAzure.CloudDrive.dll -


05-myvhdwebrole-references

    • In your ServiceDefinition.csdef, define local storage to cache the VHD locally -
<LocalResources>
  <LocalStorage name="MyCloudDriveCache"
  cleanOnRoleRecycle="true"
  sizeInMB="30" />
</LocalResources>
    • Make sure you select the 1.1 Azure guest operating system, specifically setting the osVersion in ServiceConfiguration.cscfg:
<ServiceConfiguration 
  serviceName="MyWebRole" 
  osVersion="WA-GUEST-OS-1.1_201001-01"/>
    • Make sure you define settings to access Page Blobs in Azure Storage -
<Role name="MyWebRole">
  <Instances count="1" />
  <ConfigurationSettings>
    <Setting name="DiagnosticsConnectionString" value="UseDevelopmentStorage=true" />
    <Setting name="AzureStorageAccount" value="devstoreaccount1" />
    <Setting name="AzureStorageKey" value="Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6I…" />
    <Setting name="BlobStorageEndpoint" value="http://127.0.0.1:10000/devstoreaccount1/" />
    <Setting name="QueueStorageEndpoint" value="http://127.0.0.1:10001/devstoreaccount1/" />
    <Setting name="TableStorageEndpoint" value="http://127.0.0.1:10002/devstoreaccount1/" />
    <Setting name="BlobContainePCLDrives" value="pathpcldrives" />
    <Setting name="VHDDriveName" value="pathpcldrive.vhd" />
  </ConfigurationSettings>
</Role>

Here’s the whole configuration to try this out in development:

06-myvhdwebrole-configuration

  • When writing the Web Role code that accesses the read-only files in the VHD


    07-myvhdwebrole-codesample



    Sample Code to mount a VHD using the new CloudDrive APIs


    • Initialize the Windows Azure Drive environment and the local cache
    • Do the Windows Azure dance to get a reference to the VHD stored in Azure Blob Storage
    • Create a snapshot of the existing Page Blob (VHD).
    • Mount the drive from the snapshot
    • Read the files using System.IO!
    • Un-mount the read-only cloud drive and delete the snapshot in your web role shutdown code.


02-localstorage-vhd-simulation-screenshot 
A snapshot of a VHD in a Page Blob simulation in Azure local storage

Important: in the local development fabric you don't upload the VHD -

My thread: Getting "Error HRESULT=80070003"
when creating a snapshot of a CloudDrive


http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/ad801616-9fc0-41b1-8fea-f275f6f83bb3

My thread: Read-only CloudDrive is a snapshot: when should I delete it?
http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/5f21c780-2370-4a80-a664-27cb8dcb34fa

Is it possible to Delete a blob snapshot?
http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/67c274f7-ad83-4281-8bf7-4a07503a6efb/

Azure Drive
http://nmackenzie.spaces.live.com/blog/cns!B863FF075995D18A!582.entry

Instead manually create a folder of your VHD name directory in windows explorer and copy the needed files.

Links

Beta Release of Windows Azure Drive
http://blogs.msdn.com/windowsazure/archive/2010/02/02/beta-release-of-windows-azure-drive.aspx

Windows Azure Drive White Paper

http://go.microsoft.com/?linkid=9710117

USING WINDOWS AZURE DRIVE PART 1:

MIGRATE YOUR DATA TO THE CLOUD


http://blogs.msdn.com/tconte/archive/2010/02/26/using-windows-azure-drive-part-1-migrate-your-data-to-the-cloud.aspx

My thread: Getting "Error HRESULT=80070003"

when creating a snapshot of a CloudDrive


http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/ad801616-9fc0-41b1-8fea-f275f6f83bb3

My thread: Read-only CloudDrive is a snapshot: when should I delete it?

http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/5f21c780-2370-4a80-a664-27cb8dcb34fa

Is it possible to Delete a blob snapshot?

http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/67c274f7-ad83-4281-8bf7-4a07503a6efb/

Azure Drive

http://nmackenzie.spaces.live.com/blog/cns!B863FF075995D18A!582.entry


Good times!


| More



Comments

About

This page contains a single entry from the blog posted on March 12, 2010 11:20 PM.

The previous post in this blog was Red-Gate’s SQL Compare – now with SQL Azure support!.

The next post in this blog is Running multiple apps on the iPad at the same time (Multitasking) – now!.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type 3.35