Emmanuel's Blog

Windows Azure CloudDrive
VHD – NTFS Drives in the cloud

Summary

These are general points on the Windows Azure ‘CloudDrive’ feature.  These are in no particular order – but they are important in understanding in what scenarios a CloudDrive can be useful and what potential problems you may run into.

• Windows Azure SDK 1.1 (Feb 2010) released a new feature: CloudDrive.
• When running in the fabric an Azure web or worker role can mount a VHD file - which is then available as a hard drive.
• The VHD is stored as a Page Blob in Azure Storage
• The minimum size of a VHD is 16 MB.
• The maximum size of a VHD is 1 TB (which is the maximum size of an Azure Storage Page Blob).
• You are only charged for the pages that are stored - for instance, you can allocate a 1TB Page Blob and if you are only storing 1GB of pages in the blob, then you are only charged for 1GB of storage
• The VHD must be formatted as NTFS (FAT is not supported, I have a request here).
• For smaller drives, NTFS uses up a lot of space
• 20 MB VHD: only 7.9 MB of free space after initial formatting.
• 25 MB VHD: only 12.8 MB of free space after initial formatting.
• For larger drives the additional security headers of a NTFS drive do not come into play.
• Only one web or worker role can have read/write access to a specific VHD at any time.
• You need to get a "lease" on page blob for this to work - I have not done this yet (no demo code for this).
• For read-only access, you create a "snapshot" of an Azure Storage Page Blob (VHD), which you can then mount.
• Snapshots are small in size: roughly 512 bytes, they are only a pointer to the actual page blob containing the VHD.
• Remember to delete the snapshot and un-mount the drive on role shutdown.
• The VHDs can be cached locally in your web role's local storage.
• The local "on-disk" cache maximum size depends on the VM size you chose for your web/worker role.
• Once you mount a CloudDrive you get a drive letter and you can use System.IO classes to read/write to files.
• Important: mounting a CloudDrive only works in the Azure fabric, local development fabric or Staging/Production cloud.
• You cannot mount the drive in an ASP.NET site running in IIS, a console app or Windows Forms app.
• You can only mount a CloudDrive from the same Azure storage you're running in
• If you are running in the local development fabric, you can only mount a VHD from local storage.
• If you are running in the Staging/Production cloud, you can only mount a VHD from Azure Storage.
• I have a trick I came up with if running in IIS for fast development: add the files you need to source control; create a function that returns the folder containing the files you need to access and at runtime do this:

If Not (RoleEnvironment.IsAvailable) Then

  ' So we don't get an error when running in IIS

  sFolder = System.AppDomain.CurrentDomain.BaseDirectory() 

  Return sFolder

End If

• Good overview document: "Windows Azure Drive - February 2010v2.docx" - see the "Usage Patterns and Best Practices".

Example: How to read text files
from a VHD in an Azure Web Role

In this sample projects I cover in the screencasts, we create a VHD, copy in some text files, and deploy the VHD to Azure Storage; then in a web role we mount the VHD as a CloudDrive and read the text files.

• Create a VHD

  In Windows 7 and Windows 2008 Server R2 
  you can create and mount VHDs natively 

• Can be done through code
• Can be done manually in Windows 7 and Windows Server 2008 R2 through Disk Management.
• We use Computer Management > Disk Management > Create VHD to create a 20 MB VHD.

A mounted VHD in Windows 7 looks like a regular hard drive 

• Upload the VHD to Azure Storage (Page Blob)

Sample code to get a reference to a Page Blob in Azure Storage

• Can be done through code
• See sample project 'MyVHDDrive' - references DLLs from C:\Program Files\Windows Azure SDK\v1.1\ref
• We might also be able to upload the VHD to Azure Storage through the latest version of Cloud Storage Studio or other GUI clients (download worked).
• In development the VHD storage is a simulation, so the same method cannot be used. Instead there's a trick to create the VHD and copy files manually.

• The directory is normally: C:\Users\[USERNAME]\AppData\Local\dftmp\wadd\devstoreaccount1\[CONTAINER]\[DRIVE.VHD]\[FOLDER]
• If you ran into the path max length issue, and you have defined _CSRUN_STATE_DIRECTORY, let’s say as “C:\t” then: C:\t\wadd\devstoreaccount1\[CONTAINER]\[DRIVE.VHD]\[FOLDER]




• A Web/Worker Role mounts the VHD


• In your project, add references to the Windows Azure SDK DLLs, including the new Microsoft.WindowsAzure.CloudDrive.dll -

• In your ServiceDefinition.csdef, define local storage to cache the VHD locally -

<LocalResources>
  <LocalStorage name="MyCloudDriveCache"
  cleanOnRoleRecycle="true"
  sizeInMB="30" />
</LocalResources>

• Make sure you select the 1.1 Azure guest operating system, specifically setting the osVersion in ServiceConfiguration.cscfg:

<ServiceConfiguration 
  serviceName="MyWebRole" 
  osVersion="WA-GUEST-OS-1.1_201001-01"
… />

• Make sure you define settings to access Page Blobs in Azure Storage -

<Role name="MyWebRole">
  <Instances count="1" />
  <ConfigurationSettings>
    <Setting name="DiagnosticsConnectionString" value="UseDevelopmentStorage=true" />
    <Setting name="AzureStorageAccount" value="devstoreaccount1" />
    <Setting name="AzureStorageKey" value="Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6I…" />
    <Setting name="BlobStorageEndpoint" value="http://127.0.0.1:10000/devstoreaccount1/" />
    <Setting name="QueueStorageEndpoint" value="http://127.0.0.1:10001/devstoreaccount1/" />
    <Setting name="TableStorageEndpoint" value="http://127.0.0.1:10002/devstoreaccount1/" />
    <Setting name="BlobContainePCLDrives" value="pathpcldrives" />
    <Setting name="VHDDriveName" value="pathpcldrive.vhd" />
  </ConfigurationSettings>
</Role>

Here’s the whole configuration to try this out in development:

• When writing the Web Role code that accesses the read-only files in the VHD
  

  
  

  
  

  
  Sample Code to mount a VHD using the new CloudDrive APIs



• Initialize the Windows Azure Drive environment and the local cache
• Do the Windows Azure dance to get a reference to the VHD stored in Azure Blob Storage
• Create a snapshot of the existing Page Blob (VHD).
• Mount the drive from the snapshot
• Read the files using System.IO!
• Un-mount the read-only cloud drive and delete the snapshot in your web role shutdown code.

 
A snapshot of a VHD in a Page Blob simulation in Azure local storage

Important: in the local development fabric you don't upload the VHD -

My thread: Getting "Error HRESULT=80070003"
when creating a snapshot of a CloudDrive

http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/ad801616-9fc0-41b1-8fea-f275f6f83bb3

My thread: Read-only CloudDrive is a snapshot: when should I delete it?
http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/5f21c780-2370-4a80-a664-27cb8dcb34fa

Is it possible to Delete a blob snapshot?
http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/67c274f7-ad83-4281-8bf7-4a07503a6efb/

Azure Drive
http://nmackenzie.spaces.live.com/blog/cns!B863FF075995D18A!582.entry

Instead manually create a folder of your VHD name directory in windows explorer and copy the needed files.

Links

Beta Release of Windows Azure Drive
http://blogs.msdn.com/windowsazure/archive/2010/02/02/beta-release-of-windows-azure-drive.aspx

Windows Azure Drive White Paper

http://go.microsoft.com/?linkid=9710117

USING WINDOWS AZURE DRIVE PART 1:

MIGRATE YOUR DATA TO THE CLOUD

http://blogs.msdn.com/tconte/archive/2010/02/26/using-windows-azure-drive-part-1-migrate-your-data-to-the-cloud.aspx

My thread: Getting "Error HRESULT=80070003"

when creating a snapshot of a CloudDrive

http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/ad801616-9fc0-41b1-8fea-f275f6f83bb3

My thread: Read-only CloudDrive is a snapshot: when should I delete it?

http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/5f21c780-2370-4a80-a664-27cb8dcb34fa

Is it possible to Delete a blob snapshot?

http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/67c274f7-ad83-4281-8bf7-4a07503a6efb/

Azure Drive

http://nmackenzie.spaces.live.com/blog/cns!B863FF075995D18A!582.entry

Good times!