NetworkJutsu

Network Security Consulting | San Francisco Bay Area

VMware

VMware ESXi Home Lab [2016]

By Leave a Comment

I recently bought an Intel NUC 6th generation (Skylake-based Core i3) as a VMware ESXi host. This is an addition to my existing host that I assembled back in 2012. It has served me well over the years and hope that it will continue to do so for several more years. Four years old in the computing world is obsolete, but I think it is still a very capable machine for what I use it for. Having said that, it will still be running as another ESXi host.

Related: VMware ESXi on 10th generation Intel NUC (Frost Canyon)

You might be asking why in the world would someone need two ESXi hosts at home that is not even studying for VMware certifications. I do, however, want to point out that I wanted to take the VCP certification back then that was why I attended VMware vSphere Install, Configure, and Manage [V5] class in a community college back in 2012 – check my tweet about it. Anyway, the primary reason is that the 32GB RAM on my first ESXi host is beginning to be too limiting for my CCIE lab purposes. I have to manage the amount of VMs that are turned on or I will be doing a memory overcommit.

The secondary reason, which is related to the first one, is that I really want to run vCenter Server so I could play with vSphere Flash Read Cache and other stuff. The Flash Read Cache feature was mentioned to me by @matthaedo on his response to my tweet. I did a quick search about this feature since I did not know what it was, how to do it, etc. and found out that one of the requirements to take advantage of the Flash Read Cache feature is vCenter Server, which I did not have. Yes, I could’ve installed the trial version of vCenter then test it out in my existing host and be done with it. But, I really wanted to add another host anyway so that it can handle my other VMs and add vCenter Server Appliance. Fortunately, I found out that we have VMAP (VMware Academic Program) campus subscription. One of the things that this subscription gives us is that it allows any student, staff, or faculty to use VMware infrastructure products for personal use to gain hands-on experience. The program is similar to the VMUG Advantage but only few products are included. For example, we do not have access to VMware Virtual SAN in the portal but the VMUG Advantage’s EVALExperience does. So, if one is looking for VMware licenses that are relatively inexpensive ($200) then the VMUG Advantage subscription is the way to go. I just wish it includes VMware NSX though.

Let’s go shopping

Intel NUC

At the time of writing, the Intel NUC 6th generation has four models shipping, two of them have identical Core i3 CPU and the other two have identical Core i5 CPU. The four models are NUC6i3SYK, NUC6i3SYH, NUC6i5SYK, and NUC6i5SYH. The main differences between identical CPU models are the size of the enclosure and the SATA3. Models with SYH at the end have the space for 2.5″ HDD or SSD and of course the SATA3 connector for it. The NUC6i7KYK – Skull Canyon is now available for purchase and ESXi 6.0 U2 can be installed with BIOS changes.

There have been several virtualization folks who had been running Intel NUC for years. The earlier NUCs are not compatible with the official ESXi ISO so it requires a customized ISO that contains proper drivers to install ESXi. With the 6th generation, it is no longer needed to create a custom ISO making it a seamless install.

Without further delay, here are the parts of my ESXi host:

1 x Intel NUC6i3SYH
1 x G.SKILL 32GB (2 x 16GB) F4-2133C15D-32GRS
1 x Sandisk Cruzer Blade 8GB USB Flash Drive (not pictured)
1 x OCZ Deneva 2 C Series 240GB D2CSTK251A20-0240 (not pictured)

I just happened to have the USB flash drive and OCZ SSD drive so feel free to buy the alternative parts listed below:

1 x Crucial 32GB (2 x 16GB) CT2K16G4SFD8213
1 x Sandisk Cruzer Fit 8GB
1 x Samsung 850 EVO 250GB 2.5″ SSD and/or 1 x Samsung 850 250GB EVO M.2

If you decide that you want additional NIC, then you may want to purchase the StarTech USB 3.0 NIC. Right out of the box, it won’t work but with a little tweaking then you could definitely make it work.

At this time of writing, the Crucial ($165) is cheaper than the G.SKILL ($180) so buy the Crucial instead. I just happened to get the G.SKILL a little bit cheaper than Crucial when I bought the parts. If you do not need the SSD since you have a NAS (I have the Synology DS1812+) then forget about the links above. In fact, my other ESXi host does not have an HDD or SSD installed in it. Though, that may change in the future. I already have Samsung 850 EVO 1TB SSD in my NAS to speed the VMs up but I might still buy one for Flash Read Cache.

Installation

Before installing ESXi 6.0 Update 2 on the Intel NUC, I suggest checking the BIOS version first. Mine came with BIOS version 24, which was released in Oct 2015. The new version is 36, which was released last month, can be downloaded directly from Intel’s download page. The BIOS will work on all four models so be sure to download and apply it first before doing something else. Upgrading the BIOS was very easy. All I did was download the SY0036.BIO file, copied it to a USB flash drive, and powered on the NUC with the USB flash drive in it and hit the F7 key. Then, the NUC recognized that there was a BIO file then I followed the screen prompts to install the new BIOS.

After the BIOS was complete, I head back to my other computer and tried to create ESXi 6.0U2 ISO using UNetbootin. I’ve used that in the past and have successfully installed the different OS using the bootable USB created from UNetbootin. However, this time around, UNetbootin did not recognize my USB flash drive for some reason. My Windows recognized that there was a USB flash drive but UNetbootin did not. I then decided to use Rufus and created the ESXi bootable USB flash drive and it recognized the USB flash drive just fine.

Next, I booted the NUC with the USB flash drive and entered BIOS settings to make sure that the boot order was correct so that every time that the NUC reboots, it always tries to boot from the USB. Once ESXi installer files were loaded to the RAM, I followed the screen prompts and picked the same USB flash drive I used to boot it from as the destination drive for the ESXi files.

ESXi on Intel NUC

Thoughts

As you can tell from my tweet, I’ve only had the Intel NUC for two days but I am quite pleased with it. I am not going to lie but I wish my NUC had more oomph. I could’ve gotten the Core i5 or wait for the Core i7, which in theory, should provide more oomph but stay under $500 was important. The $90 difference in Core i5 is probably not much but I’ve spent quite a bit of tech stuff for that past few months, like my router, managed switches, and 1TB SSD. If I ever need more compute power, I could use the vMotion feature to move it to my Xeon-based ESXi host and run it from there.

I also noticed that the NIC doesn’t seem to work correctly when I set it to 9K MTU. I kept getting an error message and when I did a quick search one guy said that it means that the NIC does not support the jumbo frame. However, one reader said that the NIC itself supports the jumbo frame but the driver might be the culprit. It’s not really a big deal but it would’ve been great if I could’ve used 9K MTU without doing some tweaking. The majority of my wired devices are already set to 9K so this is another odd man out.

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

VMware Certified Associate Data Center Virtualization (VCA-DCV)

By 1 Comment

I don’t follow VMware certifications so I didn’t know that they even announced the new Associate level certifications last month. I recently just found out about it and decided to give it a whirl, since it’s free – for limited time only. This new certification level doesn’t really change how the prerequisites of the upper level certifications like how Cisco designed theirs. If you are familiar with Cisco certifications, then you know that the Associate level certifications are prerequisites for majority of the Professional level certifications.

Cost

The certification will cost you $120 if you take it after the promo period. VUE and VMware are offering all Associate level certifications for free, with exception of VMware Certified Associate – Network Virtualization (VCA-NV), until September 30th. If you take it after September 30th, then it’s $60 since VUE seems to offer their promo until the end of the year. Don’t quote me on that though.

If you schedule it right now online then it’ll cost you $60, but again since VMware has a promo so you can enter the coupon code to make it free. The code you need to enter is VCA501. If you happen to try to schedule online, you may get an error after going through the motions of registering for the exam. It seems like VUE has been having this issue for quite some time now since I’ve seen postings online that people are getting the same error as I did. While you can certainly try it out, I suggest that you don’t waste your time doing so and just give VUE a call. It should be less than five minutes to schedule via phone unless there’s a long hold time.

Certification

VCA-DCV is really a sales certification rather than a technical certification in my opinion. It didn’t really ask any technical questions at all, if my memory serves me right. It just validates that you know the features well enough to answer questions of clients that are interested in VMware’s virtualization software products.

The certification doesn’t really have a prerequisite, which is a relief for people wanting a VMware certification since VCP still has the class requirement. This is most likely VMware’s response to the outcry of a lot of IT professionals. While it’s a good attempt to appease the IT professionals, in my opinion, VMware failed. This certification is worthless in my opinion. I will definitely not even put this on my resume unless it magically made a reputation that everyone has to have it. I did, however, put it on my about page and LinkedIn just for the heck of it.

How to pass VCA-DCV exam?

At the time of this writing, I only needed to take the free VMware Data Center Fundamentals class offered by VMware and was able to pass the VCA-DCV exam. However, I have to say that I have personal experience with VMware ESXi (since 3.5) and the VMware vSphere: Install, Configure, and Manage [V5] class I took also helped me to pass the exam.

The VMware Data Center Fundamentals class is a self-paced and is approximately 2.5 hours long. It is a pretty short class so one could take this at one sitting. The exam might have changed by now so I suggest that one should buy the VCA-DCV Official Cert Guide: VMware Certified Associate – Data Center Virtualization book and also take the free self-paced class from VMware, especially for people who do not have experience with VMware vSphere.

Thoughts

If you are looking for a VMware certification to add on your resume, this is not it. Don’t waste your money on it. The $120 is better spent on a VMware vSphere: Install, Configure, and Manage class that you can take from a community college instead of paying for the pricey 5-day classes offered by training partners, like Global Knowledge. I took mine from Ohlone College for less than $100. However, that pricing is limited to California residents and non-CA residents have to pay over $300 to take the class, if I am not mistaken. One of the forum members on the site that I frequently go to found a lowest cost that can be taken from the comfort of your home that doesn’t require residency to a particular state. It’s from Stanly Community College and I believe it will set you back less than $300. Yes, these colleges are authorized VMware IT Academy as you can see from this list. The list no longer includes the schools that offer the VMware IT Academy program. Check the VMware IT Academy site to search for schools near your area.

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

Nested ESXi 5.1 – Virtual ESXi In A Physical ESXi Host

By Leave a Comment

Yes, it’s another blog post about how to create a nested ESXi 5.1 for lab purposes. The reason why I decided to write one was because of my frustration with the guides out there that I tried to follow but didn’t work. Well, technically, it did work but what I wanted to do was to enable 64-bit guest OS inside the virtual ESXi host. I already had several template VMs that I can just clone from my physical ESXi to virtual ESXi host, so in theory, it should be a lot faster than building a 32-bit based VM. Well, guess what? It didn’t happen. I spent quite some time to get it to work the way I envisioned it since the guides that I followed missed the extra step. If I left it alone with 32-bit, then I would’ve probably saved time! But, it really bugged me because my ESXi host has a CPU that allows me to have 64-bit guest OS inside a virtual ESXi host, so I had to find a solution to make it work.

My ESXi host has been neglected so I was running the original ESXi 5.0 image since I built it. Yesterday, I decided to get it updated and build my virtual ESXi lab for the purpose of Nexus 1000V and probably for VCP studies. I have until the second week of December to use the exam voucher that I received from my VMware Academy course that I took last year for the VCP5-DCV exam. If you’re interested in updating your ESXi 5.0 to 5.1, please follow this guide. Now, that guide doesn’t update to 5.1 U1 so you may want to use this instead.

CPU Requirement

To virtualize ESXi, you should have the right CPU or nested virtualization will not be possible. Below are the CPU requirement:

  • Intel VT-x or AMD-V for nested 32-bit VMs
  • Intel EPT or AMD RVI for nested 64-bit VMs

Preparing Physical ESXi Host

Before creating the our vESXi, we need to edit a config file on our physical ESXi, which is the /etc/vmware/config file. There are two ways that I found on how to do this and are shown below. One way is to use echo command to append it to the config file, which is shown in the left side of the image, and the other way is to use the good old VI editor, shown on the left side. I actually prefer the latter.

config file

Note: This is assuming you’ve enabled SSH services of your ESXi host. Mine has been configured that way since I use vmkfstools command to clone VMs.

If you do not know how to enable SSH service, please use the steps below. Otherwise, please skip and go to the next step.

  • Go to configuration tab.
configuration tab
  • Click the Security Profile link and a new view is presented. Click Properties link
    security profile
  • Once Services Properties shows up, click the SSH and click Options. A new window will appear labeled SSH (TSM-SSH) Options. Click Start button. This will temporarily enable SSH until the next reboot. If you prefer to enable it completely, please feel free to tick the Start and Stop with host option and click the OK button.
    SSH

Creating Virtual ESXi 5.1 (vESXi) Host

  • Create a standard VM just like you’re used to.
  • Once you’re in the Guest Operating System section, choose Linux as the Guest OS and Other 2.4.x Linux (64-bit) as the Version. Once everything is done, do not power on the VM yet.
    Guest OS Version 1
  • Before powering up the VM, there are more settings that needs to be edited. Go to edit the VM properties and check the following: CPU needs to have a minimum of two virtual sockets or cores, at least 2GB of RAM, and of course do not forget about the ISO image on the virtual optical drive.
  • Once you’re done with the Hardware tab of the Virtual Machine Properties, go to Options tab and choose General Options settings. Change the Guest Operating System to Other and Version to VMware ESXi 5.x (experimental). If you do not see this, make sure the RAM is at least 1.x GB. Though, I recommend using 2GB of RAM since I’ve encountered issues when I changed it to 1.5 GB of RAM when booting up, YMMV.
    VM properties
  • The vESXi host is now ready to be powered on for the first time. Follow the installation process. Don’t worry if you see a warning about HARDWARE_VIRTUALIZATION. Just finish the installation and we’ll fix it later. This is the one that is missing in a lot of the guides out there so that 64-bit OS will work. It may have worked for their lab but mine didn’t work at all using their guide.
  • Once vESXi is up and running. Configure the network settings to static or use DHCP reservation. Up to you what you want to do but I like mine static since it reduces broadcast in my network. I also like to add the IP address on my computer’s host file since I do not have internal DNS server at home yet.

Hardware Virtualization Warning

As mentioned earlier, ignore the Hardware Virtualization warning since I was able to figure out on how to fix it. Follow the step in the preparing physical ESXi host section but this time, connect to your vESXi and make the change. Again, this is assuming you’ve enabled the SSH service.

UPDATE: This step is no longer needed. The preparing physical ESXi host section will do the job. Apparently, I was confused on how I fixed the issue. At first, I was using vhv.allow = “TRUE” so that’s the reason why it wasn’t working then made a lot of changes and not tracking what I’ve done.

Modifying vSphere Standard Switch

I do not really know why this is needed but I wasn’t able to start my 64-bit OS without modifying the switch settings. To modify the standard switch, please follow the steps below.

  • Go to Configuration tab. Click Networking on the left pane of the screen under Hardware. Find the vSwitch that corresponds to your Virtual Machine connection type. In my case, I have three NICs so that’s why there are two NICs on the screenshot. Once the correct vSwitch has been found, click Properties.
    vSphere standard switch
  • The vSwitch Properties window will appear. Click the LAN and click Edit. LAN Properties window will appear. Go to the Security tab and tick the Promiscuous Mode box. By default, this is set to Reject so change it to Accept. Once selected, click OK. Close all the windows and you’re done.
    LAN properties

Cloning An Existing VM

This step is optional. I’ve only included this for my notes and probably help other people on how to clone VMs on ESXi.

  • Create a new folder on your shared storage.
  • Issue the vmkfstools -i command as shown below.
    vmkfstools
  • Create the custom VM on the vESXi host and make sure to use the existing disk pointing to the newly created VMDK file.

Thoughts

This is a really great feature that VMware included on their hypervisor. Not a whole lot of people or organizations out there has the luxury of building a lab that consists of physical hardware. That said, having virtual ESXi lab allows individual and/or organization to study VMware related certifications or test before deploying to their production environment. I wish Cisco followed their suit a long time ago but much anticipated release of VIRL (Virtual Internet Routing Lab) then I can’t complain. Though, we’re still missing a lot of information about the VIRL. I know that people are saying there will be free and paid version. I wonder what will be the limitation of the free version. Hopefully, the free version will be able to create a CCIE topology so once the version 5 comes out then individuals who are studying for the lab won’t need to buy new gears and IOS 15 licenses to access all the technologies.

Reference

VirtuallyGhetto

Want to get certified?

VCP5-DCV Official Certification Guide
Mastering VMware vSphere 5
Mastering VMware vSphere 5.5
Mastering VMware vSphere 6

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

VMware ESXi Home Lab [2012]

By 1 Comment

If you’ve been following my tweets, then you know that I haven’t been doing a lot of CCIE studying. For about three weeks last month, I tried to finish my Openfiler + ESXi (vSphere Hypervisor) home project. Obviously, it wasn’t three straight weeks worth of time – a few hours per day/night. I’ve been very excited about this home project and I’ve already put off another mini-project that I have to do – which is to install my Samsung 128GB SSD drive and drive caddy to replace my optical drive for extra HDD space on my mid-2009 15″ MacBook Pro.

Anyway, if you’ve been researching for whitebox builds then you’ve probably have seen people using Mac Mini as their ESXi host. I’ve considered running ESXi on Apple Mac Mini since others have successfully done it using this guide, but I’ve decided against it. The form factor of Mac Mini is hands down a good feature to have, but there are some limitations to it that may not work for the future growth of my home network and my need to play with different technologies outside Cisco.

There is also Intel NUC as a very attractive ESXi host. However, the older generation of NUCs needs to have a custom ISO so that the NIC would work. With the new Intel NUC (6th generation), the ESXi 6.0 Update 2 works out of the box on these units. I actually talked about it here since I’ve added another ESXi host in my home lab.

Related: VMware ESXi Home Lab – Intel NUC (Frost Canyon)

Before we delve into the ESXi build, let me give you a little bit of background of my old setup. My old server was supposed to be my ESXi since the motherboard is listed here as compatible with ESXi. However, upon building it and really giving a thought it turned out to be not the best solution for what I wanted it to do. I could make it work but as what other people would say just because you can doesn’t mean you should. Having said that, I just built it as a regular Windows box with virtualization software running on top of it to run VMs with different OS (Linux, Unix, and Windows) in them. It worked great and no complaints. However, as a technology enthusiast, there was this voice that kept telling me to run VMware ESXi at home. As you can already tell, the voice won and I’ve decided to buy and build a new home server. Besides, this particular build will be used for VCP studying if ever I decide to really get serious in pursuing it. I do want to get it, but it isn’t my day-to-day job so I am still contemplating. Yes, I am well aware that you need to have instructor-led training first before you can register to take VCP. I am actually currently enrolled in a local community college which is a member of VMware Academy. This is the cheapest route for me so I decided to sign up for the class. I could’ve asked my current employer to pay for the boot camp style VMware vSphere: Install, Configure and Manage class but I’ve decided against it since it really is outside my day-to-day job responsibilities. I am just taking this class just to expand my knowledge outside the Cisco realm.

Anyway, for curious minds out there, here’s my old home server specs:

AMD Phenom II X6 1090T Black Edition
ASUS M4A88TD-V EVO/USB3
2 x G.SKILL Ripjaws 8GB
Dell PERC 5/i
3 x Western Digital Caviar Black 1.5TB
Corsair AX850 PSU

Without further ado, here are the parts of the home server:

Lian Li PC-V351B
Supermicro MBD-X9SCL+-F
Intel Xeon E3-1230 V2 Ivy Bridge
Antec EA-380D Green
2 x Kingston 8GB DDR3 SDRAM ECC Unbuffered Update: It’s 32GB RAM now.
SanDisk Cruzer Blade 4GB USB Flash Drive

This whitebox build is in the high-end range, so if you are looking for just ESXi home lab use then this may not serve your purpose. There are other builds out there that are suitable for inexpensive home lab use, like HP N40L, so choose whatever will work for you. This build will also be used for ESXi home lab (nested ESXi 5.x) and for “production” so I wanted a more powerful host than the N40L and more expandable in the RAM side of things. As already mentioned, I run different OS with different purposes in my home server to provide services for myself, friends, and family. Some of these services are the following but not limited to proxy, FTP, SSH, and etc.

If you were paying attention, the build didn’t have any hard drives in it. This build was designed to run quietly and with low power requirements. I chose to use 4GB USB flash drive to install ESXi on to save power, money, and repurpose the OS hard drive on my old server. Since I do not have CD/DVD-ROM driver either, I searched the web on how to create a USB bootable flash drive and it was pretty straightforward, just follow this guide.

At first, I used Openfiler as my NAS and it worked great. However, I am convinced that the cause for crashing/rebooting of my NAS box was because my RAID controller was overheating. Never really spent a whole lot of time troubleshooting it because this project went over the projected schedule that I wanted to spend my free time on. With that said, I just pulled the trigger and bought Synology DS1812+ as my NAS for CIFS/SMB and NFS shares that my ESXi box will use as its datastore. So far, I love this Synology NAS box and will never go back to Openfiler. Don’t get me wrong, Openfiler is really good but I believe Synology or other NAS manufacturers out there is the way to go. Yes, I could’ve stuck with Openfiler by just buying a new RAID controller to save cost initially, but over time I believe I’ll end up paying more money on energy costs by using a regular PC.

Well, I hope this blog post is a little bit helpful for the people trying to build a whitebox ESXi host. Let me know if you have any questions and I’ll try to answer them as soon as I can. Enjoy!

You might also like to read

VMware ESXi Home Lab [2016]

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.