I am fortunate enough to have two different labs in which to play. First is my Demo Lab at the Mosaic Technology office. I was granted an isolated port off our firewall directly to my lab, and a private VPN into that network. On that network I have a Dell M1000 blade chassis with two M610 blades, two R710 and one 2950 racked servers, several Dell switches and two EqualLogic iSCSI SANs. This affords me enough hardware to configure just about any possible environment or scenario physically. However, the nature of my job keeps me away from the office most of the time, with only 2-3 days per month onsite. I can still VPN into my lab and make changes as needed, test new solutions or offerings, and even integrate new hardware and products into the rack. But as we added features and products, it became less of a sandbox and more of a showcase. There wasn’t enough flexibility in the environment for me to extend my knowledge as needed.

As a result, I decided to take matters into my own hands. When it came time for me to get a new laptop, I leveraged our position as a Dell partner to select a Precision 6500 mobile workstation for my new PC. Large 17 inch display, dual i7 processors, and 4 memory slots gave me great flexibility. Additionally, there is room for a second internal hard drive. I purchased with 4 GB ram and a 250 GB SATA drive standard. I installed another 8 GB RAM to bring it up to 12 GB, and added a Seagate MomentusXT hybrid drive in the second slot.

Installation of VMware Workstation 7 gave me the horsepower I needed to build out a lab internally. To begin with, I configured the virtual networks to use two vlans, the first VLAN8 was set for NAT to the host as a LAN connection. VLAN9 was configured to be host-only for storage network. I copied over a basic Win2k3 and Win2k8R2 image, creating two VMs from the base images. On the Win2k3 server I installed Active Directory and DNS. The Win2k8R2 became my vCenter server. I then created two VMs, 5 GB in size, and installed ESXi 4.1 on each. Once configured, I joined them to vCenter. As shared storage, I chose NFS. From the Virtual Appliance Marketplace, I downloaded a Fedora VM, added a 100 GB second drive, configured it to export the second disk via NFS, and then mounted it to the ESXi hosts. I also downloaded the Celerra Simulator from EMC to act as an iSCSI target as well.

Here is what the network diagram looks like:

With this in place, I now have a vCenter lab that I can take along with me wherever I go. Since the Public network is NAT to the host, I have a virtual NIC on my laptop on VLAN8. Once the VMs are powered up, I can use the tools on my laptop to interface with the lab. I can SSH to the hosts or the NFS server, use my vSphere Client, or RDP into the vCenter server or DC as needed. This is a great little lab configuration that I carry with me all the time. If I need to troubleshoot a problem, or try out a new tool or application, I can install it into my laptop lab easily. While I won’t win any awards for performance, as everything is a VM running on a single hard drive, at least I can bring it up anywhere. On several occasions, I have fired it up onsite with a customer to try out a new configuration or demonstrate a concept. Nothing beats a live demo!

Anyhow, I hope that this inspires some of you to give it a try. Do you need to buy a couple of servers to build out a home lab? Not if you don’t want to. In most cases, a good laptop or a desktop will suffice.

]]> http://whiteboardninja.wordpress.com/2011/10/13/building-a-portable-vsphere-lab/feed/ 0 timantz virtual Network http://whiteboardninja.wordpress.com/2011/10/13/my-vcap-dca-experience/ http://whiteboardninja.wordpress.com/2011/10/13/my-vcap-dca-experience/#comments Thu, 13 Oct 2011 23:48:03 +0000 timantz http://whiteboardninja.wordpress.com/?p=76 Continue reading →]]> There is an old joke that starts out “What do you call the guy that finishes last in his class at Medical School?”  The answer is “Doctor”.

As bad as that joke is, it is kind of how I feel about my VCAP-DCA result.  In a previous post, I wrote about my VCAP-DCD testing experience.  I was pretty confident going into that test because I have been designing customer virtualization solutions for the past three years.  Design is a daily function of my job, and the DCD was a natural extension this.  However, since leaving Bowdoin College and joining Mosaic Technology, I have been removed from the day to day grind of administration of said virtual infrastructure.  And like most things in life, there is a bit of “use it or lose it” when it comes to the DCA side of things.  I have my test labs, both at home and at the office, but without daily reinforcement, things can slip.

With that in mind, passing the DCD was validation, but I approached the DCA in a different light.  I studied the VCAP-DCA blueprint, which I recommend highly.  More importantly, take a look at Sean Crookson’s VCAP-DCA index.  This not only follows the blueprint, but gives a great outline of not just the topics, but where to find the subject matter.  Read it, study it, practice it, and then do it all over again.  Take some time to review the BrownBag sessions that Cody Bunch hosts on http://professionalvmware.com/.  They were of immense help as well.  I took all of this into consideration, prepared for the exam, but went into it with low expectations.  I didn’t expect to pass, but instead get a good feel for where my weak spots were, and where I would need to improve.  Try my best, but prepare for the worst.

As to the actual exam… While I can’t divulge the questions or content, I can discuss the nature of what you will need to know in order to prepare yourself.  The exam is only 34 questions long, as opposed to the 113 questions in the VCAP-DCD, but it is a completely different format.  You are presented with a virtual lab environment, and from question 1 all the way to question 34, you are building upon that same vCenter datacenter.  Tasks you perform early are built upon as you progress through, and early mistakes can multiply quickly and spin out of control.  Some tasks are simple, such as creating a cluster, performing basic storage and VM configuration tasks, setting up switches and such.  However, the tasks become more difficult as the exam rolls along.  You will be asked to perform some operations that you may have never done before, or in a specific manner you are not accustomed to do them.  Be aware of what you are doing, and make sure that you FOLLOW ALL DIRECTIONS!  If it says to perform via vMA, don’t do it via the vSphere Client GUI!  You will notice that you can move forward and back between the questions without switching into the lab environment.  I discovered this about 10 questions in, when I needed to go back and fix something that I missed in an earlier question.  Once I did this, I quickly moved forward through the questions and wrote on my dry erase board things such as 15-performance, 16-network, 17-storage… (not real, but you get the picture).  I broke down what was needed, then went back and tackled the infrastructure pieces in order.  It was important to do this in order to ensure that I had enough time to finish the exam.  I then went back and performed the tasks that were ancillary, such as generating reports, logging, making specific changes, etc.  Core infrastructure first, data and tweaks later was my mantra.   I don’t know if it was a weighted score, but if it all works at the end, you must have done something right.  As it was, I did not complete all of the tasks. I hoped that my jumping around would count for something.

As far as what to expect, all I can say is know everything on the blueprint.  More importantly, if there is a task on the blueprint, make sure you know how to do it not only from the GUI, but via command line, vMA, and if possible…PowerCLI.  Don’t underestimate your ability to perform operations via command line.  Know how to do it without a GUI, and you will drastically improve your chances to pass the VCAP-DCA.

And in reference to the joke at the top of this post… I passed my VCAP-DCA exam on the first try.  With a passing score of…300!  Exactly what was needed to pass.  Therefore, as with number 100 out of a class of 100 at medical school, all that matters is that we both passed.  I am now certified as a VCAP-DCA to go with my VCAP-DCD.  Next on board is preparing a VCDX design submission, and hopefully defending in Frankfurt in February, 2012.

With this in mind, the first step was to get my VCAPs out of the way first.  I decided on the VCAP-DCD first, as my focus these days tends to be more on designing new environments or upgrading existing installations of vSphere.  My administration skills are good, but without a functioning production environment to maintain on a daily basis, I can only beat up on my test lab so much.  I figured that leading with my strong suit would give me an advantage.

The first step after scheduling my exam was to ask my good friend Google if he had any references out there that have been through the DCD before.  Forewarned is forearmed.  Some good sources come from Eiad Al-Aqqad, Sean Crookson, and Gregg Robertson to name a few.  And, of course, the VCAP-DCD Blueprint!  While the NDA prohibits us from discussing the actual information on the exam, I can pass along some of the info regarding the setup and organization.  Most of this info is out there either in blogs or on the official VMware websites, so I believe I can speak freely.

To begin with, this exam is NOT to be taken lightly.  Most of the recommended study information is the same being offered for the VCP exams.  While this information is important, it is not really what this exam is about.  Where the VCP was very technically detailed, focusing on numbers such as maximum memory, hosts, CPUs, and other bits of esoteric knowledge, the VCAP-DCD had very little to do with it.  Rightly so, the powers-that-be assume that since the VCP is a requirement you’ve already memorized it and they want to test your brain in a different way.  The VCAP-DCD is all about is design.  They want to know how you think when you are putting all of the pieces together.  It is good to know the maximum number of hosts in a HA/DRS cluster, but more importantly, given a set of customer requirements… How would you design an environment?  What kind of decisions would you make, and why would you choose A over B?  Man on second with no outs in the 8^th… do you sacrifice him to third, and why?  Carolina whole pig BBQ or Texas beef ribs?  Those kinds of decisions are what you will be faced with during the –DCD, only about virtual environments and how to build/create them.

With several years under my belt designing customer environments, I was comfortable with the questions and my answers as the test went along.  Technically, my hands-on experience carried me through the exam.  If you are looking for a class or book to give you the answers you seek, you are looking in the wrong place.  Course work and memorization will get you only so far with the VCAP–DCD.  Without a good working knowledge of vSphere and experience with design decisions, you will be at a disadvantage.

The actual questions and answers were not the difficult part of the exam.  The time limit was the greatest challenge.  This exam is 113 questions long, with 5 design questions in the mix.  Those 5 questions are a huge hurdle.  If you haven’t done it yet, make sure you use the VMware VCAP4-DCD Design Tool Simulator before you take the exam.  These questions have a Visio-type interface, and without prior experience with the tool, you will waste valuable time familiarizing yourself.  These design questions take 10-15 minutes each, and with only 225 minutes for the whole exam, you are down to a little over a minute per question for the rest of the exam.  There are several ‘drag and drop’ questions that can be time sinks as well if you are not careful.  In a nutshell, you need to stay on task and mind your time during the exam.  Don’t get hung up on a difficult question, flag it for review and move along.  If you get through in time, go back during the review period and make your decisions.  If you don’t make it to the end, at least you will get through more questions.  I can’t state this enough… you need to make quick decisions.  I have a sneaking suspicion that one of the factors you are being judged on is not only your knowledge, but you quick you think on your feet (so to speak).  You either know it or don’t, and putting you in a pressure situation such as not having enough time to think things out will weed out some folks that don’t have the base design experience.  I was able to make it through with 9 minutes to spare, and didn’t run into any overly difficult time-sink type questions.  There is about a minute or so after the exam where I stared at the blue background screen… and then the window pops up congratulating me for passing the VCAP-DCD.  I got a 321, with a 300 being a passing mark.  I took a deep breath, not realizing until after that I had been holding my breath in anticipation.  With the VCAP-DCD behind me, I am now studying for the VCAP-DCA.  With any luck, I will get through it and begin my design submission for a VCDX defense in February.

Last summer, EqualLogic released their Hybrid PS6000XVS.  This array combined eight 100GB SSD drives and eight 600GB 15k SAS drives within a single 16 drive enclosure.  A proprietary RAID type allowed each volume to be stretched across both sides of the array, both SSD and SAS.  The two sides of the array have different I/O and performance profiles, and the EqualLogic controllers are aware of the differences in capacity and performance.  When data is written to the Hybrid array, incoming data pages are tagged as ‘hot’ by the system.  This is a rating of relative activity of pages within the array.  Recent reads and writes to the data marks the page as ‘hot’.  As read/write activity tails off, the page becomes ‘cool’ and even ‘cold’ if the page hasn’t been accessed recently.  Write activity automatically mark a page as ‘hot’.

Since the EqualLogic controllers know that the SSD drives have a higher performance profile than the 15k SAS drives, the data is written to the highest performing area within the array, which is the SSD drive set.  Continued reads will keep the data page ‘hot’, but inactivity for a specific page will cause it to ‘cool’ off relative to other pages.  Once a threshold is reached within the array, the data page is then moved (on the fly) over to the 15k SAS side of the array but still within the same volume.

This process continues to occur as new data is written to the volume, leveraging the high performance of the SSD drives and the higher capacity of the 15k SAS drives to optimize data page placement within the volume.  If a data page that resides within the 15k SAS side of the volume is accessed for reads or writes, the temperature of that data page increases.  Once the temperature of a page on the 15k SAS drives rises above a page that resides on the SSD side, the two pages are swapped (on the fly), therefore optimizing data page placement within the volume.

As you can see above, the ‘colder’ pages move to the lower tier of storage, while the ‘hotter’ pages are migrated to the highest performing level.  This optimizes data placement within the array and grants SSD level performance for all of the data within the volume while extending capacity with 600 GB SAS drives.

When considering VDI rollouts and the increased performance demands that a View infrastructure requires, it is easy to see that the PS6000XVS Hybrid array is a welcome addition to the storage toolkit.  By leveraging a Hybrid array, we can easily provide +30 thousand IOPS within a single enclosure with a couple TB of capacity.   This is a game changer in the VDI world, and one you should consider, especially if you are using EqualLogic storage.  If you are deploying VDI without considering one of these Hybrid arrays, Caveat Emptor.

The company develops graphical software for the engineering industry. Having successfully virtualized the datacenter with vSphere they’re currently considering some sort of virtual desktop initiative. Being very sensitive of their intellectual property and with remote development sites, there are concerns with not only moving data large distances but also losing control of those resources. The developers typically rely on high-end systems that leverage GPU cycles on graphics cards, and therein lays the question – in the words of their systems administrator:

Knowing what we know about ESX, since it’s doing all of the hardware abstraction for the OS’s, even if you had a great video card in a host, it would never see it or utilize it because vSphere only presents the generic VGA experience with emulated OpenGL, etc. My question to you is, does View work differently with this? How can I bring high-end graphics to an end user with View?

The Connection Manager performs several functions in a View environment. It coordinated the building and configuration of the client pools by integrating vSphere and Orchestrator to create VMs, Linked Clones, and pools of desktop resources for consumption by entitled users. The most prevalent paradigm is the base desktop image with thin clones, connected to thin clients at the end points. These VMs are good all around performers, but not really suited for high end graphics. The VM uses a generic VMware graphics adapter, with usually no more than 128 MB video RAM. Good for office workers, but not intense graphics workloads. There is another way to get a high-end user experience using View. You just need to think of virtual desktops, not just virtual machines.

Another primary function of the Connection Manager is… well, to connect clients with desktop resources. The Connection Manager is a Broker. It is the proxy that connects an end point running the View Client software, such as a Thin Client, workstation, or laptop to an OS that is running the View Agent. This client request is brokered through the View Manager and establishes the relationship between client device and desktop resource. While in most cases that desktop resource is a VM, it can also be a Terminal Server session or a physical PC it these have the View Agent software installed.

When looking at it from a performance perspective, a standalone desktop with the View Agent installed allows a client to leverage all of those physical resources through the View infrastructure out to the end point. You could have a farm of high end workstations in your datacenter, and provision them to clients all over the world. All of the heavy lifting and rendering would be done locally, and the end user would see the end result on the thin client. . Blade PCs, intended for desktop users, can also be provisioned via View in this manner also. You could look at these for dedicated desktops, or even purchase some Dell Precisions loaded with memory and graphics cards, and stand them up on racks in your datacenter. People could connect via a thin client from anywhere using View and have access to physical resources.

More importantly, the PC is in the datacenter. This means that all of the software, rendering, CPU horsepower, and more importantly, the intellectual property of the company are retained in the datacenter. This protects both the hardware and software corporate assets. There is a much higher level of security within your environment when you can retain control of all assets within your datacenter.

This question spurred a great deal of discussion internally, with fantastic results. Hopefully this will get some of you thinking of View as not just VMs connected to Thin Clients, but as more of a true Virtual Desktop solution for the workplace.

First of all, I want to be very clear that I believe backups are the ‘killer app’ for deduplication, and moving forward, file sharing as well.  That being said, there are a couple of ugly truths that nobody in the dedupe business talks about.  The one we will discuss here is Exit Strategy.

When considering deduplication as part of an upgrade or change in your current backup design, there are several options to weigh.  Software or Hardware, integrated or stand-alone, and then the various offerings for each type.  There are several to choose from, and factors surrounding the decisions that you make.  I could go deeper into each of them, but lets not do that today.  I can save that for another day.

When it comes down to your choice, you will most likely have a simple disk-to-disk (D2D) target for your backup vendor of choice, where the save sets are stored in a compressed and deduplicated manner.  However this is done, either by the backup software or the target device, there is metadata associated with this backup that stores the compression algorithm and the deduplicated block index.  From an external ‘point of view’ the data looks normal, but is stored in the compressed/deduplicated format on the file share based on the metadata.  This is pretty much standard across the board, and is the magic behind the curtain.

Here is where it gets tricky.  You can have multiple dedupe engines working on the same piece of data.  For example, if your backup application can dedupe and compress while writing to a D2D target, it has a ‘point of view’.  If your backup application doesn’t do dedupe, the D2D target could be on an appliance such as Data Domain, which natively performs deduplication inline on anything being stored within, creating another ‘point of view’.  You probably aren’t going to dedupe at multiple layers, but it is possible due to the separate ‘points of view’ established.  You will need to be careful when extracting data by following the same chain back out of the dedupe string to get back to the flat files that you backed up originally.

As an example, you could use your backup software to write your save sets to a D2D target, and it will write that data in a way that is compatible with the backup application.  If that D2D target happens to be a volume on a Data Domain appliance, the data written into the device will be deduplicated inline as it is written to disk, therefore being reduced in size significantly.  Those save sets, and others added afterward, are contained in the same ‘point of view’ allowing you to store significantly more data than physically available on the disks natively.

Now we come to the ugly side of dedupe, which is what to do when the honeymoon is over.  How do you break up with a dedupe device?  The customer in question was a long-term DD customer, and was facing a large increase in maintenance fees post-EMC acquisition.  The cost/benefit had fallen to the point where they wanted to go in a new direction.  They wanted to know if they could just export the data on the Data Domain box to another location.  I wish it was that easy, but alas… it was not meant to be.

As I mentioned earlier, to get out of a dedupe data set, you need to unwrap the data in the same manner that you put it into the store.  The customer needs to open up their backup application, browse to the save sets, and clone them off to another location.  This location could be tape, another dedupe device, or a simple D2D target.  In their case, they had been cloning out monthly sets to tape for archiving purposes, and simply needed to step up the process for weekly and daily tapes until their migration to the new backup application was completed.

In conclusion, deduplication is a great thing for storage and backup vendors, but it can create additional complexities when you try to migrate away from a specific technology.  Make sure you ask about exit and upgrade strategies while shopping around for your next backup appliance.  It could save you headaches down the road.

600 (reads) + [400 (writes) * 2 (RAID Penalty)] = 1400 IOPS.

What this means in practical sense is that to support the 1000 IOPS at 60/40 on RAID 10, you need a disk array capable of 1400 IOPS.
Take that further and calculate out a RAID 5 array.

600 (reads) + [400 (writes) * 4 (RAID Penalty)] = 2200 IOPS on the back end.
RAID 6 will look like this;

600 (reads) + [400 (writes) * 6 (RAID Penalty)] = 3000 IOPS,

which is over twice as much I/O as required by the RAID 10 array.
With all of this IOPS information at hand, we can now calculate what will be needed as hardware to support the virtual desktops.  How many disks, and what type, will be needed to provide the required 1000 IOPS?  Will 8 * 15k-SAS drives in RAID 10 suffice or will we need have to go with 28 * 7.2k-SATA drives in RAID 5?  Possibly it will come down to a mix of the two types.  And yet another card to play is Solid State Drives.  Enterprise SSD drives can support IOPS at 4000 per disk, which can be a perfect match for some VDI deployments.
Now that you understand the relationship between IOPS on the desktop and back end storage requirements, what is the best way to figure out exactly what you will need for YOUR virtual desktop project?  The best tool that I have found is Stratusphere from LiquidwareLabs.  The LWL server is an OVF, downloadable and installable into your existing VMware infrastructure.  From that point, we generate an installable agent that can be deployed on your existing desktops.  Once installed, the agent collects performance information from each of the desktops relating to CPU, memory, network, and disk, from a user and application level.  Once collected it is sent back to the LWL server for analysis.  After several weeks of collection, we can start to see a usage profile for your users, based on time of day, applications used, and overall functionality of each desktop.  With this data in hand, we can then profile your user base and size your desktop needs.  While the average user consumes 10 IOPS, your users may only require 6 or 7 IOPS on average, with a subset of users that require 12-15 IOPS.  We can also track application usage and determine how best to deploy applications to virtual desktops, either installed in the base image or ‘ThinApped’ and streamed to the clients as needed.
So what have we learned today?  First, that it is very important to plan ahead when considering a VDI deployment, and that benchmarking and analysis with tools such as LWL’s Stratusphere can help you to profile your infrastructure.  Secondly, that we will need to correctly size our back end storage support for any VDI deployment, and why it is so important to success.  And third, that when it comes to hard drives, size really DOESN’T matter, at least not when concerning IOPS!  It is all about controller type and spindle speed.