My Planned VHD Organization

In my last post I talked about wanting to move my entire computing environment over to VHDs.  Not just a development environment or test environments but everything.  This would include a general work VHD for my wife and I, a video production VHD for all my video stuff, a production development VHD, various server VHDs (mostly for use during development using Virtual PC) and whatever test VHDs I want.  I would no longer be booting into a standard operating system installation as we have been since the beginning of personal computers.

Some of you might be thinking,”Why in the world do this?”  Well, the last time I had to rebuild a computer it took approximately 2 days.  This included installing the O/S, MS Office, developer tools, all my utilities, plug-ins, and all the updates.  This is wasted time.  Plus, there are often times that I’ve loaded a tool I wanted to check out only to find out it has hosed something of my system.  Maybe it’s not something critical but it’s enough to force me to spend a few hours trying to weed out what files or settings got changed.  If I could test out tools in a exact and isolated environment just by copying my current VHD how incredible would that be?  Plus, assuming I backup my VHDs regularly, let’s say I got a really nasty virus or something that wiped out my system.  No big, I just delete that VHD and restore it from the backup.  Since we’re talking about an entire VHD it’s just a file copy, not a restore that takes hours.  Talk about system restore. :) 

So, as usual, before I start researching the details of how things work my wheels started spinning faster than a hamster running from the cat.

I heard about differencing disks and started looking into them.  Differencing disks allow you to create a “parent” VHD and a “child” VHD.  For instance, a common example is testing how your website looks with different versions of IE, which cannot be installed simultaneously.  You create a parent VHD with the operating system and whatever other software you want in the base.  You then create several child differencing VHDs that reference the parent.  Each child has a different version of IE, so one with IE 6, one with IE 7 and one with IE 8.  The child VHDs are called differencing disks because they only contain the information that is “different” than the parent.  You can also use a child differencing disk as a parent to another differencing disk, thus chaining them.

This really got my heart going.  I instantly thought of a grand hierarchy design like the following:

VHD_Original_Layout

The base Win7 VHD is the only true parent.  All the others are child differencing disks.  I could update the base Windows 7 VHD with whatever updates Microsoft would throw out and all the other VHDs would get it.  All bottom level child VHDs (video, production, test, etc) would have Office, Windows Live, etc.  It seemed like a perfect environment.

Here’s the catch.  A parent VHD cannot be changed.  If you do so, the child differencing disk will be corrupted (because the differences no longer are current) and you will loose the child VHD and all data in it.  Microsoft even recommends that you set parent VHDs to read only to protect against inadvertent changes.  Wow, that’s a real bummer.  I believe some of the enterprise level virtual vendors let you do scenarios like this across serves, thus making it easy to deploy updates and all, but that doesn’t help me.  I want to run native VHDs on my Win7 machine.  Oh well.  Scratch that.

I think technically this could be set up just for the sheer exercise, though I won’t bother to take the time to try it.

So, now that reality has stepped in how do I plan to set this up?  I’ll have a separate standard VHD per environment with no differencing disks.  A great tip from Stephen Rose during his Virtualization 101 for Developers presentation was to make copies of your base VHDs, mark them read only and put them in a backup folder.  That way if you ever want to create a new one you don’t have to start from scratch. 

So, I plan to create my base Win7 image.  I’ll put a read only copy of that into a VHD Backup folder.  I’ll then install Office, Windows Live, etc onto that image, and put a read only copy of that into the backup folder.  At this state this is my general use VHD.  I’ll make a copy of that, rename it as my video production VHD and install my video production software on that.  I’ll make a read only backup of that as well incase it gets hosed or I want to try out some fancy new video software in the future without harming my current VHD.  I’ll make another copy of my general use VHD, rename it to my production development VHD and install all my development software.  You get the idea.

When you boot off of a VHD you always have access to the base C drive (which will be a different drive letter).  So, as far as document storage there are details there I’ll have to work out.  I don’t know how security is worked out but I doubt I’ll be able to access documents in my original My Documents folder.  I’ll have to see what the best practice is out there.  I’ll probably have a generic data storage folder on the C drive that I’ll keep all of my documents in.  I only have one drive in my laptop so I can’t throw everything onto a D drive like my desktop.

Just a few tips to leave you with:

  • Backups are important!  VHDs can get quite large and so are my video files.  While most other documents are small and I can use some sort of cloud storage for my documents or versioning system for my code this won’t save me the time and hassle of losing my VHDs.  I have yet to see an online backup provider like Mozy or Carbonite that has the upload bandwidth I would require for something like this.  I’ll plan on having a couple of eSATA drives that I can back up to on a regular basis.
  • You can’t boot from a VHD on an external drive.  That’s just the way how Microsoft’s I/O mini-port driver works.  They probably did this to make sure you can’t set your laptop to boot from something that might not be there.  Plus, if your external drive got disconnected for any reason, such as a loss of power or your USB cable falling out, it might instantly corrupt your VHD.
  • If there is not enough drive space to expand your entire VHD you can’t boot from it.  For instance, let’s say you have 250GB available on your hard drive and you create VHDs that can expand to 150GB.  If over time you get to a point where you don’t have at least 150GB available to the VHD (including the current physical size of the VHD) you can’t boot into it.  Until you can get rid of some data that VHD will be effectively unbootable.  This is probably a good reason to keep your existing O/S on your system rather than having a pure VHD system. 

I’ll keep updating how it goes. 

Converting my entire production environment to VHDs

Ever since virtual PC type systems have been out, long has been the dream to run any type of operating system you want, multiple operating systems at the same time, all working independently of each other.

However, only in the last year or two has this really become a reality with the latest hardware supporting virtual environments natively.  Now Windows 7 lets you boot off of a VHD as if it was a standard hard drive making full use of your actual hardware devices.

After attending Stephen Rose’s presentation on Virtualization 101 for Developers my eyes were suddenly open to the possibilities of what could be done.  I could move my entire development environment to a virtual PC image.  This would allow me to copy that image at any time allowing me to test beta products, new upgrades, etc without worrying about damaging my current environment.  Many times I have upgraded to new releases (or betas) that caused previous programs to stop running, Dlls required in legacy projects to be removed, etc.  In some rare cases (SQL Server 2008 RC2) I had to reload my entire environment because I could never get my machine back to its original state.

Now that Windows 7 allows you to boot from a VHD my wheels are suddenly turning.  I could move my entire computer use (not just development) to VHDs.  This would allow me to have a base image for general tasks like email, MS Office, etc that my wife and I could use for general day to day tasks.  I could have another environment for all my video editing work, which is especially useful since video compression codecs can sometimes cause havoc.  I could have a production development environment and as many beta environments as I want.  If I want to download a 30 day trial of any product I just create a copy of the appropriate VHD, boot off of that and try out the product.  After 30 days if I don’t like it I just delete the testing VHD and it will have never touched my production VHD.  If I do like the product I just delete the testing VHD, purchase the full copy and install it to my production VHD.  Very nice and clean.

I just purchased a new HP dv7 laptop with the Intel core i7.  So, this is the perfect time to get started.  While it is possible to wipe the drive clean and use VHDs without any core operating system at all I’ll leave the HP environment intact.  That way I can always fall back to that just in case.  If for some reason my sound stops working I doubt HP will be willing to care if I can’t reproduce it in the standard OS.

So, I’ll document how I set this up and how it works out in future blog posts.  Enjoy the ride!

Netflix and the Extinction of DVDs, et al

It was recently announced that Netflix will add streaming to the Nintendo Wii as one of its capabilities.  They already stream to personal computers, the Microsoft Xbox, Sony Playstation and various small devices.

As we just received a Wii for Christmas I am excited to try this out.

As a technology geek I have long thought out how to move my entire media collection (photos and personal videos, music and DVDs) to a computer in our house that we could watch from any tv or listen on any stereo.  This is already easily done but the funds are a little out of reach for our growing family and it certainly isn’t available for the masses.

That’s where Netflix comes in.  They revolutionized the DVD rental industry when they allowed you to rent through the mail with no late fees.  While I was dubious at first at whether their business model could really turn a profit with that much overhead they have done an incredible job.  I’m really glad they have made it through the roughest part.

When Blockbuster saw the competition Netflix was creating they started their own DVD rent by email service, however, they one upped Netflix by allowing you to return them back to your local store.  With Netflix you had to send your DVDs in before they mailed you your next batch.  With Blockbuster, you could return your DVDs back to any local store and rent another right there.  No waiting.

Around the same time Apple thru iTunes, Amazon and Netflix allowed you to download your movies.  However, this was really confined to users who watched on their computers or had laptops or iPods connected to their TVs.  This was a very small market mostly consisting of consumers with technical knowledge able to afford the equipment or college students.

However, then comes Netflix streaming content to the Xbox, Playstation and now Wii.  Suddenly they have an audience that is already familiar with playing rich content on their own devices.  Console gaming devices are almost as much of a part of any home entertainment system as is a DVD player or stereo system.  These days connecting gaming systems to the Internet is a simple task. 

In my opinion, Netflix is the leader in bringing true streaming movie content to the general masses.  While cable and satellite companies have been offering this for years it just hasn’t really caught on.  In talking with my friends, we all have cable or satellite offering the feature, but we just don’t use it.  I don’t know if it is the limited available content or the pricing.  But we just don’t use it.

Another incredible thing that Netflix has done is completely change the overhead they are required to keep.  As they move more and more of their customers to streaming content, their assets will be completely digital available anytime and anywhere in an instant.  There will be no need to ship physical and fragile media all over the country amounting to an incredible cost in shipping time, warehouse inventory, staff, etc.

By giving the general media consumer instant access to movies on general devices they already own Netflix is pushing the DVD out to pasture and really making content over the Internet a reality.

There will always be consumers who prefer to own physical media, but the writing on the wall is even more pronounce than ever before.