Season of Change

PernixData logo   Fall is one of my four favorite seasons. As the days grow cooler and shorter, the leaves start their color change. The yellow, orange, and red leaves leave the New England hillsides looking like they are on fire. The holiday season kicks off with Pumpkin Spice coffee and beers hitting the market. As Summer winds down into Fall, the season of change means different things to each one of us.
Today, I am turning over a new chapter in my life. After over six years as a Solutions Architect at Mosaic Technology, I am moving on to something new. Starting today, I am joining PernixData as a Systems Engineer. I will be focused on business opportunities in the Northeastern US and Canada. I’m very excited to join an amazing team at PernixData. The first time I saw FVP in action, I got excited. This technology changed the way I looked a things in a way similar to the first time I watched Mendel Rosenblum vMotion a virtual machine between hosts on stage at VMworld. From that moment on, I was hooked. When the opportunity to join the team at PernixData presented itself, I could not turn away.
I cannot say enough about the six years I spent at Mosaic Technology. I learned a great deal about this business and leveraged those skills and abilities I honed there to achieve my VCDX. While I am sorry to leave my Mosaic family, I am excited about the new opportunities in front of me with PernixData.
Change is all around us. You embrace it and take on the challenge, or you are swept under as it rolls along. I, for one, am enjoying the ride up front.

Posted in Uncategorized | 4 Comments

Review: Troubleshooting vSphere Storage by @mwpreston

A must-have reference for vSphere Admins

In my 10+ years of managing and designing vSphere environments, I have come across a few books that should be on the desks of every vSphere Administrator. You should not be caught without a copy of Mastering VMware vSphere by Scott Lowe and Nick Marshall, VMware vSphere Design by Scott Lowe and Forbes Guthrie, and vSphere Clustering Deepdive by Duncan Epping and Frank Denneman. Mike Preston’s Troubleshooting vSphere Storage could easily be added to the list.
While the book is only 5 chapters long, it is organized in a logical fashion that not only allows for easy reading, but quick bookmarking. The first chapter gives an overview of how storage is organized within virtual environments. It lays out the framework of storage organization within vSphere and how it relates to the physical world of servers, networks, and shared storage. Some basic information is introduced for the novice, followed by more detailed explanations of how the pieces interact. The usual ‘alphabet soup’ of acronyms doesn’t detract from the layout and keeps the reader on track. With this foundation in place, it allows for the other chapters to flow in an organized fashion.
The following chapters outline which tools to use to analyze your storage, where to look for problems and how to identify them. Common problems such as contention, capacity, and overcommitment are identified for the reader and several tips are given to show not only what these problems will look like in your environment but also how to identify and resolve them.
The three appendices are the ‘Crown Jewels’ of the book. Appendix A contains a list of your most common storage troubleshooting steps, followed by the commands and procedures you will need to take in order to eliminate each one. Knowing which questions to ask are good, but having the commands at your fingertips when you need them can be invaluable when problems arise. Preston has saved your bacon with this section. Appendix B lists out all of the ESXTOP commands and functions, which ones you will need for various issues, and how to use them. This will save time and frustration for those SysAdmins that don’t frequently use the tool. And finally, Appendix C lists out all of you iSCSI error codes. This is something that most of us will seldom need for day-to day administration. However, that one or two times when you are scratching your head trying to navigate out of a difficult situation, it can save critical time.

Troubleshooting vSphere Storage is a great reference for any vSphere SysAdmin. Not only does it outline an excellent troubleshooting methodology for your environment, but also lays down a strong background for readers to understand just why storage is important for a healthy vSphere design. I would recommend this book to anybody that manages vSphere as well as Storage admins that need to support vSphere.

Posted in Review, storage, vSphere | Tagged , , , | Leave a comment

Paying it forward

Last week, Scott Lowe posted “Putting the User back in VMware User Group” In that post, he offered to mentor 5 users interested in presenting at their local VMUG group for the first time. This offer was extended to help the regular guy, a VMware customer and VMUG attendee that has never presented, but wants to give it a shot. Cody Bunch followed suit and posted on the #vBrownBag that he would mentor 5 users as well. I applauded Scott’s effort and would like to follow suit with Cody.
I am offering to mentor up to 5 courageous people that want to step up to the podium and speak at a local VMUG or VTUG meeting. This is for first-time presenters willing to step up and take on the challenge. What does this mean to you, and what can I help you with?

- Coordinate efforts with your local VMUG leaders
– Help selecting a topic on which to speak
– Guidance and feedback on content and format.
– Review and provide feedback/suggestions for improving your presentation.
– Provide tips and tricks for public speaking.
– Do a dry run via WebEx or a Google+ hangout.

If you think you are ready for it and are willing to take myself (or Scott, Cody and the others) up on the offer, take the first leap. Send me an email as well as reach out to your local community leader. I’ve already had one person take me up on the offer. Don’t miss out on the chance to share with the #vCommunity.

Posted in Public Speaking, VMUG, VTUG | Tagged , , | Leave a comment

VCDX: Second time’s the charm

It has been almost 3 years since I first set my eyes on the VCDX certification. I moved from being a VMware customer to a Partner 2 years earlier, and had my ‘Design Mojo’. I was ready to step up to the next level of certification and show what I could do on a larger stage. However, there were a few obstacles in the way. I needed to obtain both my VCAP-DCA and VCAP–DCD certifications to qualify for the VCDX. And of course there was work, life, marriage, and all of the other things I needed to deal with on an ongoing basis. Fast forward to August, 2012 and you would find me typing furiously during the week prior to VMworld. The submission deadline for Barcelona-VMworld Europe defenses was that Friday night at 3:00AM (eastern). I worked tirelessly to get my application completed and fell short. I missed the deadline and vowed not to miss another.

I took a month off and began again in preparation for the PEX13 defense. I made it that time and was lucky enough to score an invitation to defend in February. I studied my design constantly, preparing for the defense. However, I did not do a ‘Mock Defense’ or ask for feedback from my peers. I was pretty secretive and wasn’t interested in feedback. Those decisions hurt me significantly, as I was not prepared for interaction as part of the defense. Additionally, going in ‘cold turkey’ was a source of apprehension. I think I set myself up to fail by trying to do the whole thing alone. In the end, I did not make the grade at Partner Exchange.

After another month off, with black rain clouds circling my head, I got back into the swing of things and started again. I took the feedback that was offered by the VCDX Defense Panel to heart and began redesigning my application. This time I did things different. I participated in 4 Mock Defenses. I acted on a Mock Panel for 2 others. I sent my design to peers for review. More importantly, I took good notes on every question that was asked in both Mock defenses and Peer reviews. I went through the entire list of questions and wrote out my answers to each question in line. If the answer was complex or needed more background, I would find a blog post, whitepaper, or book citation to list within my answer and link to the site.

As part of my study program, I reviewed my design, I reviewed my presentation, and most importantly, I reviewed my question list and answers. Where each question had a link, I followed it and reviewed the source. While I needed to know my design and presentation in precise detail, it was my question list that prepared me the most when it came to my state of readiness. If you can answer questions from your peers regarding a design, you gain a level of comfort and preparation you cannot reach by simple review. Take the time to see something from another point of view, and you open your mind to new possibilities.

With a renewed sense of enthusiasm and a better approach to preparation, I went into my VMworld13 VCDX Defense full of self-confidence. My preparation strategy was validated when I walked out of the Defense room. I felt that I had nailed it. Regardless of the outcome, I was 1000x better prepared for my second defense than my first run. In the end, I was granted VCDX #112.

With all of this behind me, my advice to prospective VCDX candidates is to take the time to involve yourself with your peers. Ask your peers to review your design. Do plenty of Mock Defenses. If you don’t have other VCDX candidates in your area, use WebEx or Google Hangouts to do your Mocks. Set up a rotating schedule where one person defends and the others act as panelists. Do one defense each night for a week, so each person gets a dedicated night to their design and input. Ask for criticism, and take it to heart. Nobody should design in a vacuum. Interaction with your peers will help your preparation better than anything else. And most of all, “Just Do It.”

Posted in certification, VCAP-DCA, VCAP-DCD, VCDX, vmware | Tagged , , , , | 10 Comments

VCDX Preparation: Q & A

Since I am currently undertaking the VCDX Certification process, many people ask me questions regarding preparing for the Defense. Yes, I capitalize ‘Defense’ because it’s a big deal to me right now. I have the benefit (?) of having gone through an unsuccessful Defense attempt at PEX in February ’13. Hopefully I can pass along a few things I’ve learned the hard way to other prospective VCDX candidates.

What was the biggest misconception you had going into the process?

My biggest misconception was that the defense would be all about the overall design. The VCDX is really about the ‘Design Process’, not the actual design. They don’t care if you have 10, 20, or 100 hosts in your overall design. They are really interested in what approach you took to gathering the project requirements and constraints, what assumptions you have taken, and if you have appropriately identified the risks involved with the design. Then, how did you determine the various components of your design, why you made decisions within that design, and how those decisions impact the overall design. What are the implications of changing any of the parameters, and how would you alter your design based on those changes? Why did you make this choice? What if you didn’t have Constraint-A, or Requirement-B? Where is the documentation to support that choice?

What size environments are you typically designing, what do you think the “minimum” would be?

This one is easy. “Size matters not.” Actually, the size of the design is not really as important as the process involved in determining the size of the environment. Personally, mine is 10 hosts, but was technically challenging due to several strict constraints. It was accepted for a Defense… twice. As I mentioned above, the VCDX Application is more about process and procedure, methodology, and the approach you took to the design. Your design needs to be big enough for you to show competency in the process and methods of designing a vSphere environment. I am assuming that if you’ve gotten to this point in the VCDX certification process, you know your vSphere down cold. You need to be technically sound and very good at what you do. You don’t necessarily need to show the panel that you can design a 250 host multi-site infrastructure, but instead show the process involved in getting to the point. Why did you need 250 hosts? How did you come to those conclusions? Where did you place them, and why?

How long did it take you to write your first VCDX design? How has that changed for subsequent designs?

My design is a partially fictitious one, based on a real customer, but adding a few pieces from other customers to fill it out into a ‘holistic’ design. I had all of the pieces, but had trouble with the documentation and template forms. Over 6-9 months, I wrote and re-wrote it several times. Finally, I settled on a template I was happy with and got down to writing. I spent about 45 hours writing before I finally missed the deadline for submission at Barcelona’12 last year. I took a few months off, then revised my design for PEX’13 and only needed another 35 or so hours to wrap that one up. Bear in mind, I already had the ‘real’ design in place and just needed to write up the Defense design. This time, I used a different template/form, borrowing from the SET for some inspiration. This re-work of my design only took about 24 hours to clean up and revise before submission. Overall, I spent about 100 or so hours over 14 months preparing two VCDX Applications.

How do you write up a design for submission? What sort of format are they asking for?

This is the most difficult piece of the design, because nobody gives you a template to follow. By the time you decide to attempt the VCDX, you should be well onto the path of doing designs for customers and such. The ability to design an infrastructure shouldn’t be the hard part. What drove me crazy for almost 9 months was the format/template of the design documentation. I started, stopped, and re-wrote my design about 5 different times before I settled on a form that I was happy with for my PEX ’13 defense. Since then, I took a look at the Plan and Design template that can be found in the SET (Solution Enablement Toolkits) on VMware’s Partner Central site. I used my original design layout, complemented it with what I found in the P&D, and created something that flowed much better for my VMworldUS’13 Defense.

Short answer: Get the SET documents and use them as a guideline. Your design should contain more than what is shown in the Plan and Design template, but it is a good starting point for the layout.

 

Once you are invited to defend, what’s next?

Prior to my first defense, I was extremely busy with work and didn’t focus as much as I intended originally. I didn’t do any mock panels, nor did I let anybody look at my design for comments. I intended to do both, but never got around to it. Life and work just got in the way of my plans. Upon arrival at PEX, I got together with @kennyga (defending) and we both did a mock defense in front of friends. My lack of preparation showed, and I spent that night reworking my presentation and memorizing everything in sight. Obviously, it wasn’t enough and I failed to gain my VCDX.

Take the time to do a few Mock defenses. The discussions that come up can do nothing but help you prepare for something you never thought of. If a panelist sees a problem that you overlooked, it can be disastrous. However, if it was uncovered in a Mock, you can prepare for that contingency. Also, don’t be too shy about asking friends or associates (current VCDX holders?) to review your design and offer comments. While not as ‘real’ as a Mock, they also help prepare you for other contingencies. Forewarned is forearmed.

 

What’s it like in the Defense?

Unfortunately, there is a huge NDA regarding what goes on in the Defense. I like to say that the VCDX Defense is like “Fight Club”… The first rule about the Defense is ‘You don’t talk about the Defense’. We can’t really say much other than our impressions. Fortunately, the VCDX Bootcamp lets us all have insight into the process and procedures. If you are interested in attempting a VCDX defense, I strongly encourage you to attend one of the Bootcamps. They are invaluable and accurately portray how things are handled in the room.

‘Pearls of Wisdom’ regarding the VCDX Defense:

  • It was one of the most difficult and emotionally draining experiences I have gone through.
  • All of the anxiety was self-inflicted, as I became intimidated by the reputations of my panelists.
  • Don’t try to ‘bullshit’ your way out of a corner. There’s a reason the panelists are there: They are VCDX’s and know their shit.
  • Stick to your guns and don’t ‘waffle’. If you made a decision based on a constraint or requirement, justify it.
  • Don’t let yourself get psyched out by preconceived notions of the Defense Panel. They’re not there to ‘screw you over’, but instead to make sure that you know what it takes to be a VCDX.
  • Everybody on the panel has had to go through the same thing you are doing right now.
  • They all know you are nervous, and want to see how you react under pressure and scrutiny.
  • To quote Frank Herbert, “Fear is the mind killer…” http://en.wikiquote.org/wiki/Dune
  • The defense is not about determining how good your design is. If your design wasn’t correct, it never would have made it to the Defense. It is about your choices in the design, how you came to those conclusions, and why you made those choices. In short, the Defense is about YOU, not about the validity of your design.

 

Conclusions

As of right now, I am still pursuing my VCDX. I will make my second Defense appearance in a few weeks at VMworldUS’13, hopefully with better results than last time. If this information helps somebody else in their VCDX journey, it was well worth the effort. Good luck!

Posted in certification, VCDX, vmware, vSphere | Tagged , , , | 4 Comments

Mixed EqualLogic and MD3200i design for vSphere storage

Over the years, I have come across several SMB customers running vSphere on iSCSI.  While a majority of them opt for EqualLogic storage, some have existing MD3000i/MD3200i SANs that have been the workhorse Dell storage solution for years.  One question I that comes up often is ‘Can I run both EqualLogic and MD3200i SANs for vSphere?’

There is a simple way to configure both MS3200i and EqualLogic storage in the same iSCSI network while utilizing only 2 physical NICs for SAN traffic on each vSphere host.

 

iSCSI SAN Switch Configuration

  • Create 5 Vlans on your redundant SAN switches.
    • vlan10- EqualLogic
    • vlan20- MD (A)
    • vlan30- MD (B)
    • vlan40- MD (C)
    • vlan50- MD (D)
    • All ports designated for vSphere host connectivity should be trunk ports with all iSCSI Vlans.
    • Ports designated for EqualLogic should be access ports on VLAN10- EqualLogic.
    • Ports designated for MD3200i should be access ports on the appropriate VLAN.

vSphere vSwitch Configuration

  • Create 6 x vKernel ports for iSCSI connectivity
    • vmk01-vlan10
    • vmk02-vlan20
    • vmk03-vlan30
    • vmk04-vlan10
    • vmk05-vlan40
    • vmk06-vlan50
    • Bind vmk01, vmk02, and vmk03 to pNIC1
    • Bind vmk04, vmk05, and vmk06 to pNIC2

Configure storage as per Dell MD3200i Deployment Guide for ESXi 5 http://dell.to/SYILt6

Configure storage as per EqualLogic Configuration Guide http://bit.ly/16NNR4N

Mixed iSCSI Networking

 

Results:

This will offer the best redundancy available while utilizing only 2 pNICs for iSCSI storage.  If your hosts have additional pNICs available, you can break out the EqualLogic ports (vmk01 and vmk04) onto two pNICs.  On the other 2 pNICs, bind vmk02 and vmk03 to pNIC3 and vmk05 and vmk06 to pNIC4.  Of course, you could always put them on separate pNICs if you have the spare capacity.

Image | Posted on by | Tagged , , , , , , , | Leave a comment

Dell VRTX- First Impressions

On Tuesday, 6/4/13, Dell unveiled its new PowerEdge VRTX converged system during the opening keynote at the 2013 Dell Enterprise Forum in San Jose, CA. The VRTX comes in a Tower or Rack mounted form factor covering 5RU. Inside, you will find slots for 4 of Dell’s M520 or M620 blades, the same servers found in their M1000 Blade Chassis. An onboard CMC provides iDRAC and remote management for the chassis. Each blade maps up to 4 GigE ports on an integrated switch, and there are 8 additional PCIe slots that can be mapped directly to the blades for more IO and accessory options such graphics adapters. In addition to the shared PCIe slots, there is a shared PERC controller (SPERC) that controls access to up to 25 integrated 2.5 inch or 12 3.5 inch drives. Both Howard Marks and Kevin Houston have done great posts on the details and specifications of the VRTX.

Being a vSphere-centric kind of guy, my first thoughts about the VRTX were that it could replace the commonly deployed “3-2-1″ vSphere install (3 hosts + 2 iSCSI switches + 1 EqualLogic array) for SMB and ROBO scenarios. If you had up to 4 blades, integrated networking and storage, in a whisper quiet (and it is VERY quiet while running) box with integrated remote management and monitoring, why wouldn’t deploy a VRTX instead of 6 pieces of hardware?

After several inquiries relating to the shared PCIe slots and storage, I came up with the following answers from the Dell engineers onsite:

Networking: While there are only 4 onboard GigE ports per blade, you can easily add a 10Gig, FC, or CNA into the PCIe slots and map the card directly to a blade. Add 4 x dual port CNAs, map them to the blades, attach to an external switch, and carve up your network as needed.

Storage: This became a challenge. Apparently all 25 disks on the SPERC can be either directly mapped to individual blades, joined into RAID groups, and have virtual disks carved out of the RAID groups, which could also be mapped to one or multiple blades. In practice, it is similar to carving up storage in a MD1200 shelf. Where is became challenging was when I asked if there was a way to provide a shared disk for a vSphere installation. I asked several (more than 10) Dell engineers onsite at the #DellEF, and never got a definitive answer on that. Apparently nobody thought to qualify the VRTX as a vSphere platform. The answer provided was that the SPERC could create a shared disk, and present it to multiple blades via a common SCSI bus. Not FC, Not iSCSI, not NFS. The simple question of “Can you install a vSphere cluster on the VRTX” was answered with “I would imagine it would be possible.”

In conclusion, I was very impressed with Dell’s new converged infrastructure foray, the VRTX. The ability to put 4 servers, networking, and storage into a single 5U box will shake up the industry, especially if the price point is as competitive as Dell traditionally leads with. However, the storage subsystem is not much more than a DAS shelf for the blades. I think it was a great first move, but I would have loved to see an integrated MD3200 or EqualLogic controller to give the storage a little bit of intelligence. There are small form factor EqualLogic controllers currently in use on the PS-M4110 array. As far as installing vSphere on the VRTX, I’m not sure if it will work right out of the box. However, with the use of a VSA such as Nexenta to control the storage and make it available to all blades, this could be a huge success in the SMB/ROBO virtualization market. Well played, Dell. Well played indeed.

For more information on the Dell VRTX, head to the Dell TechCenter Blog, where Peter Tsai has aggregated the latest information.

Posted in blades, converged, PowerEdge, storage, virtualization, vmware, VRTX | Tagged , , , , , , | 3 Comments