New Lab Server

Here I am, procrastinating on other stuff, to talk about the new lab setup.  I promised in the last video, I’d do a writeup, and I figured “no time like the present”!

So what do I have going on?  I’ve ripped out the ML370 and ASUS RS500A boxes, and replaced them with a veritable steal from the Dell Outlet.  I decided to go all-in for a nested lab, since I can’t come up with a good reason to put together a physical lab.

So I found a Precision T5610 with a pair of Xeon E5-2620v2s.  Twelve hyper threaded cores of processor get up and go.  The Scratch and Dent unit I bought had 32 GB of RAM installed, along with a 1 TB spindle.  Not a bad start.  But I had gear to work with, and needed more RAM.  

So I ordered a nice 64 GB upgrade kit from Crucial (well, 4 16 GB kits, technically, hoping it’d give me a 96 GB box to work with.  The pre-installed RAM and the Crucial RAM didn’t play too nicely together (Windows wouldn’t load from the spindle, nor would the ESXi installer start).  Boo.  So I pulled out the factory RAM, ran memtest86 for about a day, just to be safe, and am proceeding, for the moment, with 64 GB.

Still just a start, though, as I needed storage.  I had an Icy Dock 4-bay SATA chassis and an IBM M1015 SAS RAID controller in my little HP Microserver.  That box didn’t need those things anymore, so a transplant was necessary.  4 screws later, I had lots of room for 2.5” SATA drives.  I already had a pair of 120 GB Intel 520 SSDs in Icy Dock trays, and I pulled the 2 1 TB Crucial M550 SSDs from the ASUS box.  Now I have plenty of lab storage.  If I need more, I can always take the performance hit and attach something from the DS412+ (like I’ve already done with my “ISO_images” NFS share.

**EDIT** I ran into another snag.  The LSI controller and Icy Dock combination seem to not be playing nicely with the host.  The SSDs seem to randomly just drop offline periodically, which makes me sad.  It could be a power thing (these big SSDs are kind of notorious for power problems, especially in small drive chassis or NAS units), but I’m not going to push it any further.  I pulled the controller and drive cage out of the system, and I should have a SATA power splitter after UPS shows up today (I was too lazy to leave the house LOL), so I’m just going to run the SSDs off the on-boad SATA controller channels, and be happy about it.  

At this point, I thought I was ready for ESXi, but upon installation, I hit a snag.  The T5610 has an Intel Gigabit NIC.  As such, I expected no issues, but the 82579 isn’t noticed by ESXi 5.5u2 for whatever reason.  No biggie – this thing has a nice tool-less case, and less than a minute later, I had a dual-port Intel NIC installed and ready to go (82571EB if you’re curious).  One more reboot and ESXi was installing.

Everything’s pretty happy at the moment.  Here’s what it looks like now that I’ve built up all my virtual ESXi hosts:

Screenshot 2014 09 24 08 47 22


Screenshot 2014 09 24 08 48 53


Oh, and the “Remote_External” cluster is a set of ESXi virtual machines I have running in VMware Workstation on a Precision M4800 laptop.

I’ll follow up with some network details shortly, since that’s also gotten _way_ complex recently.  All in the name of scenario-based play with NSX. More fun later!


Still here. Or “Meandering thoughts about the lab that time forgot”

So, I do still exist.  I am still here, and look at this!  I even update once in a while.

My focus has shifted a little bit over the past few months.  I’m working a lot with NSX, and have been teaching the NSX: Install, Configure, Manage for a few months now.  Like, a lot.  Q3 has been absolutely nuts for me, especially with the announcement of NSX 6.1 at VMworld.

So, what does all that mean for me, and this blog, exactly?

Well, for the blog, it means a little bit less vSphere stuff, and a little bit more networking.  

Which leads us to “What does this mean for me?”.  Well, I’ve got a stack of Cisco gear here in the office now.  That must mean Cisco certification prep.  Someday, I’ll have a break and be able to actually do that 🙂

So, what Cisco stuff is in the pipeline?  CCNA – Routing and Switching is first.  that’s the basic stuff.  CCNA Data Center is soon to follow.  I’ve been tossing around CCDA because design is just a thing I do.  Do I pursue those to the NP/ND level?  I don’t know.  My guess is no, but I never leave anything off the table.

I’ve also got the VCP-NV on my list, and the forthcoming VCIX-NV when it launches.  I’ve taken a swing at the VCP-NV – I did that while I was out at VMworld, but I just missed the passing mark.  I was caught completely off-guard by some of the stuff in the exam.  It was my own fault, however, as I spent a grand total of zero time preparing and studying for the exam.  I even talked to the cert devs about the Exam Blueprint, that I didn’t actually read until the evening _after_ I wrote the exam.

That’ll be easy to rectify, however.  I know what I need to study to bump my score up over the pass mark.  Now I just need what everyone needs – time.

I also have some new lab gear incoming.  I caught a steal from the Dell Outlet the other day.  I have a scratch & dent T5610 with dual 6-core Xeon E5s on its way, and an additional 64 GB of RAM just showed up on my front porch today.  I’ll talk through the build I’m planning as I’m working through it.  I figure with 24 threads and 96 total GB of RAM, I should be able to run many, many nested ESXi hosts, especially with the VMware Tools fling and the MacLearn dvfilter fling for nested ESXi.  My old ASUS RS500A ran pretty well with 2 6-node clusters and a 4-node management cluster, each nested ESXi host with 8 GB of RAM each.  And the RS500A only had 64 GB of RAM.  

So I’m going to consolidate the ASUS and the old ML370 into a single, more modern (and likely more power efficient) box.  The tradeoff is that I’m giving up the iKVM and iLO capabilities of my existing hosts.  It’ll be ok.  I guess I’ll just have to look at IP KVMs now.  

In other news, I’ve moved my edge to higher-end gear.  Nope, nothing so fancy or cost-prohibitive as Cisco, but definitely powerful.  I picked up a Ubiquiti Networks EdgeRouter Lite the other day, and it’s pretty slick.  I’ve got a complementary UniFi AC on the way, and that should be waiting for me when I get home.  The EdgeRouter is pretty slick.  EdgeOS is based on Vyatta Core 6.3 (just before the Brocade acquisition), so I’m familiar, in passing, with the OS.  This thing is fast!

So this will all end up in a home network / lab series as I do more cool stuff with it.

As it stands, I gotta get back to class – we’re just about done deploying Controllers…

Where has 2012 gone?!

Well, this is probably one of the most delayed “Welcome to the new year” posts so far.

I have been _busy_.  This is one of my few quiet weekends since, well, I’m not sure I remember, but even this weekend’s almost over.  I mentioned on Google+ that I’m taking my VCAP5-DCD tomorrow.  I’ll talk about that experience later 🙂

After that, I fly to California for a couple of days of team meetings.  I guess I’ll get to see what our new leadership has in store for us crazy instructors this year.  I’m definitely looking forward to seeing my fellow instructors, and meeting the new management, including the Delivery Manager I’ve been working with for almost 75 days now!

Life and work have both been keeping me hopping, as evidenced by my lack of content here.  I’ve got all kinds of ideas, drafts started, thoughts jotted down.  Maybe I’ll get to them, maybe you’ll just have to come to one of my classes to get some of my insights (which, arguably, aren’t all that insightful – I’ve been cognizant of the fact that every day I’m an instructor is one more day deeper into that ivory tower of technology).  But I still try to share things that are important to me.

So keep me in your RSS feed, I’ll drop something interesting in once in a while, but this year is getting off to a rather slow start in the same way last year sort of tapered off to nothing.  I’m trying here! 🙂


Is scripting an important skill?

So, yesterday, a colleague posed an interesting question. “Is scripting an important skill for a system administrator”?

I’d like to answer that question with a very resounding “YES!” Frankly, I have to question whether one is truly a system administrator without the skill to write a script (even if it’s simply a quick hack involving shell redirection). This is not about any particular scripting language. bash, ksh, Powershell, it doesn’t matter. Scripting is scripting. They’re all just different roads that lead to the same place: automation.

Perhaps my strong feeling about this come from my days as a Unix administrator (Solaris, primarily, but I have run Linux systems and dabbled in HP-UX). As an old-ish Unix jockey, my mentors always reinforced a common theme: if you have to do it more than once, script it! This mantra is not just because we can do more with less by way of a script, but is also for one’s sanity, really.

Let me share an example. A number of years ago, I was hired as a Unix admin for a rather large telecommunications organization. I joined their managed hosting group, and was expecting a reasonably boring life of MACs (Moves, Adds, Changes, for those unfamiliar with the acronym) – shuffling user accounts and data, adding users, groups, software, changing things on a request basis. But upon walking in the door on Day 1, they sat me down with the existing backup guy. 3 months later, this guy was no longer there, and it was just me running the backup system (we ran the backup infrastructure entirely on Solaris at that point). What’s worse, I was a rookie backup guy, and now I had 12 data centers to deal with, and I was running solo. Sure, I could have pointed and clicked my way through the X-Windows interface to the backup software, gotten RSI on my mousing arm, and gone absolutely mad knowing I wasn’t going to get any real help. But I’m all about working to live, rather than living to work. So I spent a little time learning the CLI underpinnings of my backup software, and everything that could be done (quite a bit more than via the GUI, mind you). Sure, my daily care and feeding fell behind for a little bit, but it was oh, so worth it in the end.

So I started building scripts. Reporting scripts, maintenance scripts, add a backup job scripts. If there was something that had to be done daily, I scripted something out. Weekly? Script. Monthly? Script. You get the idea. And then I loaded up my crontab in each datacenter, and I realized how much time I had to actually learn about backups and tape. All of a sudden, I had time to work with my counterparts in our engineering team to help shape the backup infrastructure. I could share pain points with them, and have data to back them up rather than anecdotes. We could dive into the real problems while I have nicely automated everything behind me. I also took the opportunity to start working much more closely with our FC SAN guys, which is a move that shaped my career for years after. Now, 6 months after I did all that work, I managed to start getting a team built up under me. Was the work all for naught? No way. Their lives were also made a bit easier as well. Were I better about my own personal data backups, I’d still have those scripts today 🙂

The moral of the story: Automation is a good thing. Not because we’re letting “the man” win and doing more with less. That’s just a side-effect. The real win for automation is that it frees us up from repetitive tasks so we can spend time doing more valuable things. And value is still what it’s all about. When your manager asks, in your annual review, “What value have you brought to the company this past year?”

Your answer could be “I worked long hard hours pointing and clicking my way through to make sure the bases got covered.” But wouldn’t you rather say “I automated the daily care and feeding of these 4 infrastructure applications my team owns, and have been working with {other team} to help them increase their efficiency by automating applications X, Y, and Z”?

I know which I’d choose.

A New Year

And hopefully that means I can start working a little harder on this blog. I never make any guarantees, but it’s possible 🙂

This past year has been a wild ride. Professionally, I got out of consulting and joined VMware Education so I can teach full-time. There were changes internally shortly after I joined that changed my perspective about that a little – ultimately good, once the dread of change washed over me and passed. Essentially, it means I’m much more busy with class delivery, which I am very much good with.

Personally, I have tried (very unsuccessfully) to help my better half sell her house. The problem we ran into is that we just can’t compete with bargain-basement prices on foreclosed homes in the area. We’re going to have to continue on carrying two households until the real estate market resolves itself. How long will that be? Your guess is as good as mine!

I also stopped smoking analog cigarettes this year thanks to personal vaporizers (also known as PVs or electronic cigarettes). Sure, anti-smoking folks still see them as terrible as analog cigarettes, but if you check out this study, there’s evidence mounting that PVs most definitely less bad for everyone. Being a longtime smoker, I’m used to the anti-smoking vitriol, and still saddened that it’s become a moral crime. I’ll be talking, from time to time, about my vaping hobby/habit, so if you’re unfamiliar with PVs, feel free to ask questions!

I broke down and jumped in with the iPad crowd, and I can’t be happier. I use my laptop less and less frequently these days because of it. Expect more to come on that front as well.

As for the next calendar year, well, we’ll see. I tend to not set too many specific goals right away, thanks to the ease of becoming overwhelmed. Professionally, I’m gunning for my VCAP-DCA and VCAP-DCD this year. 25% of that goal was passed on 12/30 when I actually sat the DCA exam. Work was driving me to that, as I will have to be DCA certified to continue to teach the vSphere Troubleshooting and Performance classes. I should hear shortly as to my DCA fate. The DCD certification is more of a preemptive strike, as it would make sense that we will begin to require it of our instructors to continue to deliver the Design Workshop, which I also currently teach. When I get the results from the DCA, I’ll post more about it and my experiences.

On the personal front, we’re trying our hands at the property rental game, just to offset the cost and vacancy of Jen’s house. Maybe we like it, maybe we run screaming into the night after the first lease expires. We’ll just have to see.

I’d say this blog will be changing, but that would be redundant, as it never particularly coalesced into anything over the course of the past year. So maybe I can get this blogging thing nailed down this year!

Thanks for reading!

How does life get in the way?

Well, not much to report these past few weeks. Team meetings, classes, and most time-consuming, getting a house up on the market has been getting in the way of content for my little, unknown slice of the blogosphere.

Nothing has been forgotten, and for the moment, unfortunately, things like this have to be placed on the back burner for a short period.

This week is last-minute touches on the house, and studying for the Enterprise Administration Exam, which I am scheduled to pass Saturday. I’ll report back on how that goes.
I would like to report, though, that a very good friend and colleague of mine has joined me on this blog. Joe has a fantastic ITSM background, and thinks a lot about virtualization in that frame of reference. Look for some amazing thoughts to roll out of his head!

In the mean time, I hope you stay tuned and be patient. The next few weeks should be better.

Delays, delays, and delays

Well, I’m still working on the vApp post, but it’s been a little while since I’ve put anything online.

The lab is, naturally, in a pretty constant state of flux, and I’ve dramatically minimized and consolidated my hardware. But that’s having a little bit of an adverse affect on performance (what a shocker, right?!). The impact is much greater than I had originally anticipated, but that should be resolved later today when a new machine will appear when the UPS guy magically appears at my doorstep 🙂

Add to that some messy travel back home from Alabama, family time, and home demolition this weekend, I’ve been a little tied up away from the lab.

So keep your eyes on this spot, a post with actual content will be coming soon, I promise!