• Skip to main content
  • Skip to footer

NetworkJutsu

Networking & Security Services | San Francisco Bay Area

  • Blog
  • Services
  • Testimonials
  • About
    • About Us
    • Terms of Use
    • Privacy Policy
  • Contact Us

Virtualization

Nested ESXi 5.1 – Virtual ESXi In A Physical ESXi Host

08/18/2013 By Andrew Roderos Leave a Comment

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Yes, it’s another blog post about how to create a nested ESXi 5.1 for lab purposes. The reason why I decided to write one was because of my frustration with the guides out there that I tried to follow but didn’t work. Well, technically, it did work but what I wanted to do was to enable 64-bit guest OS inside the virtual ESXi host. I already had several template VMs that I can just clone from my physical ESXi to virtual ESXi host, so in theory, it should be a lot faster than building a 32-bit based VM. Well, guess what? It didn’t happen. I spent quite some time to get it to work the way I envisioned it since the guides that I followed missed the extra step. If I left it alone with 32-bit, then I would’ve probably saved time! But, it really bugged me because my ESXi host has a CPU that allows me to have 64-bit guest OS inside a virtual ESXi host, so I had to find a solution to make it work.

My ESXi host has been neglected so I was running the original ESXi 5.0 image since I built it. Yesterday, I decided to get it updated and build my virtual ESXi lab for the purpose of Nexus 1000V and probably for VCP studies. I have until the second week of December to use the exam voucher that I received from my VMware Academy course that I took last year for the VCP5-DCV exam. If you’re interested in updating your ESXi 5.0 to 5.1, please follow this guide. Now, that guide doesn’t update to 5.1 U1 so you may want to use this instead.

CPU Requirement

To virtualize ESXi, you should have the right CPU or nested virtualization will not be possible. Below are the CPU requirement:

  • Intel VT-x or AMD-V for nested 32-bit VMs
  • Intel EPT or AMD RVI for nested 64-bit VMs

Preparing Physical ESXi Host

Before creating the our vESXi, we need to edit a config file on our physical ESXi, which is the /etc/vmware/config file. There are two ways that I found on how to do this and are shown below. One way is to use echo command to append it to the config file, which is shown in the left side of the image, and the other way is to use the good old VI editor, shown on the left side. I actually prefer the latter.

config file

Note: This is assuming you’ve enabled SSH services of your ESXi host. Mine has been configured that way since I use vmkfstools command to clone VMs.

If you do not know how to enable SSH service, please use the steps below. Otherwise, please skip and go to the next step.

  • Go to configuration tab.
configuration tab
  • Click the Security Profile link and a new view is presented. Click Properties link
    security profile
  • Once Services Properties shows up, click the SSH and click Options. A new window will appear labeled SSH (TSM-SSH) Options. Click Start button. This will temporarily enable SSH until the next reboot. If you prefer to enable it completely, please feel free to tick the Start and Stop with host option and click the OK button.
    SSH

Creating Virtual ESXi 5.1 (vESXi) Host

  • Create a standard VM just like you’re used to.
  • Once you’re in the Guest Operating System section, choose Linux as the Guest OS and Other 2.4.x Linux (64-bit) as the Version. Once everything is done, do not power on the VM yet.
    Guest OS Version 1
  • Before powering up the VM, there are more settings that needs to be edited. Go to edit the VM properties and check the following: CPU needs to have a minimum of two virtual sockets or cores, at least 2GB of RAM, and of course do not forget about the ISO image on the virtual optical drive.
  • Once you’re done with the Hardware tab of the Virtual Machine Properties, go to Options tab and choose General Options settings. Change the Guest Operating System to Other and Version to VMware ESXi 5.x (experimental). If you do not see this, make sure the RAM is at least 1.x GB. Though, I recommend using 2GB of RAM since I’ve encountered issues when I changed it to 1.5 GB of RAM when booting up, YMMV.
    VM properties
  • The vESXi host is now ready to be powered on for the first time. Follow the installation process. Don’t worry if you see a warning about HARDWARE_VIRTUALIZATION. Just finish the installation and we’ll fix it later. This is the one that is missing in a lot of the guides out there so that 64-bit OS will work. It may have worked for their lab but mine didn’t work at all using their guide.
  • Once vESXi is up and running. Configure the network settings to static or use DHCP reservation. Up to you what you want to do but I like mine static since it reduces broadcast in my network. I also like to add the IP address on my computer’s host file since I do not have internal DNS server at home yet.

Hardware Virtualization Warning

As mentioned earlier, ignore the Hardware Virtualization warning since I was able to figure out on how to fix it. Follow the step in the preparing physical ESXi host section but this time, connect to your vESXi and make the change. Again, this is assuming you’ve enabled the SSH service.

UPDATE: This step is no longer needed. The preparing physical ESXi host section will do the job. Apparently, I was confused on how I fixed the issue. At first, I was using vhv.allow = “TRUE” so that’s the reason why it wasn’t working then made a lot of changes and not tracking what I’ve done.

Modifying vSphere Standard Switch

I do not really know why this is needed but I wasn’t able to start my 64-bit OS without modifying the switch settings. To modify the standard switch, please follow the steps below.

  • Go to Configuration tab. Click Networking on the left pane of the screen under Hardware. Find the vSwitch that corresponds to your Virtual Machine connection type. In my case, I have three NICs so that’s why there are two NICs on the screenshot. Once the correct vSwitch has been found, click Properties.
    vSphere standard switch
  • The vSwitch Properties window will appear. Click the LAN and click Edit. LAN Properties window will appear. Go to the Security tab and tick the Promiscuous Mode box. By default, this is set to Reject so change it to Accept. Once selected, click OK. Close all the windows and you’re done.
    LAN properties

Cloning An Existing VM

This step is optional. I’ve only included this for my notes and probably help other people on how to clone VMs on ESXi.

  • Create a new folder on your shared storage.
  • Issue the vmkfstools -i command as shown below.
    vmkfstools
  • Create the custom VM on the vESXi host and make sure to use the existing disk pointing to the newly created VMDK file.

Thoughts

This is a really great feature that VMware included on their hypervisor. Not a whole lot of people or organizations out there has the luxury of building a lab that consists of physical hardware. That said, having virtual ESXi lab allows individual and/or organization to study VMware related certifications or test before deploying to their production environment. I wish Cisco followed their suit a long time ago but much anticipated release of VIRL (Virtual Internet Routing Lab) then I can’t complain. Though, we’re still missing a lot of information about the VIRL. I know that people are saying there will be free and paid version. I wonder what will be the limitation of the free version. Hopefully, the free version will be able to create a CCIE topology so once the version 5 comes out then individuals who are studying for the lab won’t need to buy new gears and IOS 15 licenses to access all the technologies.

Reference

VirtuallyGhetto

Want to get certified?

VCP5-DCV Official Certification Guide
Mastering VMware vSphere 5
Mastering VMware vSphere 5.5
Mastering VMware vSphere 6

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

VMware ESXi Home Lab [2012]

09/12/2012 By Andrew Roderos 1 Comment

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

If you’ve been following my tweets, then you know that I haven’t been doing a lot of CCIE studying. For about three weeks last month, I tried to finish my Openfiler + ESXi (vSphere Hypervisor) home project. Obviously, it wasn’t three straight weeks worth of time – a few hours per day/night. I’ve been very excited about this home project and I’ve already put off another mini-project that I have to do – which is to install my Samsung 128GB SSD drive and drive caddy to replace my optical drive for extra HDD space on my mid-2009 15″ MacBook Pro.

Anyway, if you’ve been researching for whitebox builds then you’ve probably have seen people using Mac Mini as their ESXi host. I’ve considered running ESXi on Apple Mac Mini since others have successfully done it using this guide, but I’ve decided against it. The form factor of Mac Mini is hands down a good feature to have, but there are some limitations to it that may not work for the future growth of my home network and my need to play with different technologies outside Cisco.

There is also Intel NUC as a very attractive ESXi host. However, the older generation of NUCs needs to have a custom ISO so that the NIC would work. With the new Intel NUC (6th generation), the ESXi 6.0 Update 2 works out of the box on these units. I actually talked about it here since I’ve added another ESXi host in my home lab.

Related: VMware ESXi Home Lab – Intel NUC (Frost Canyon)

Before we delve into the ESXi build, let me give you a little bit of background of my old setup. My old server was supposed to be my ESXi since the motherboard is listed here as compatible with ESXi. However, upon building it and really giving a thought it turned out to be not the best solution for what I wanted it to do. I could make it work but as what other people would say just because you can doesn’t mean you should. Having said that, I just built it as a regular Windows box with virtualization software running on top of it to run VMs with different OS (Linux, Unix, and Windows) in them. It worked great and no complaints. However, as a technology enthusiast, there was this voice that kept telling me to run VMware ESXi at home. As you can already tell, the voice won and I’ve decided to buy and build a new home server. Besides, this particular build will be used for VCP studying if ever I decide to really get serious in pursuing it. I do want to get it, but it isn’t my day-to-day job so I am still contemplating. Yes, I am well aware that you need to have instructor-led training first before you can register to take VCP. I am actually currently enrolled in a local community college which is a member of VMware Academy. This is the cheapest route for me so I decided to sign up for the class. I could’ve asked my current employer to pay for the boot camp style VMware vSphere: Install, Configure and Manage class but I’ve decided against it since it really is outside my day-to-day job responsibilities. I am just taking this class just to expand my knowledge outside the Cisco realm.

Anyway, for curious minds out there, here’s my old home server specs:

AMD Phenom II X6 1090T Black Edition
ASUS M4A88TD-V EVO/USB3
2 x G.SKILL Ripjaws 8GB
Dell PERC 5/i
3 x Western Digital Caviar Black 1.5TB
Corsair AX850 PSU

Without further ado, here are the parts of the home server:

Lian Li PC-V351B
Supermicro MBD-X9SCL+-F
Intel Xeon E3-1230 V2 Ivy Bridge
Antec EA-380D Green
2 x Kingston 8GB DDR3 SDRAM ECC Unbuffered Update: It’s 32GB RAM now.
SanDisk Cruzer Blade 4GB USB Flash Drive

This whitebox build is in the high-end range, so if you are looking for just ESXi home lab use then this may not serve your purpose. There are other builds out there that are suitable for inexpensive home lab use, like HP N40L, so choose whatever will work for you. This build will also be used for ESXi home lab (nested ESXi 5.x) and for “production” so I wanted a more powerful host than the N40L and more expandable in the RAM side of things. As already mentioned, I run different OS with different purposes in my home server to provide services for myself, friends, and family. Some of these services are the following but not limited to proxy, FTP, SSH, and etc.

If you were paying attention, the build didn’t have any hard drives in it. This build was designed to run quietly and with low power requirements. I chose to use 4GB USB flash drive to install ESXi on to save power, money, and repurpose the OS hard drive on my old server. Since I do not have CD/DVD-ROM driver either, I searched the web on how to create a USB bootable flash drive and it was pretty straightforward, just follow this guide.

At first, I used Openfiler as my NAS and it worked great. However, I am convinced that the cause for crashing/rebooting of my NAS box was because my RAID controller was overheating. Never really spent a whole lot of time troubleshooting it because this project went over the projected schedule that I wanted to spend my free time on. With that said, I just pulled the trigger and bought Synology DS1812+ as my NAS for CIFS/SMB and NFS shares that my ESXi box will use as its datastore. So far, I love this Synology NAS box and will never go back to Openfiler. Don’t get me wrong, Openfiler is really good but I believe Synology or other NAS manufacturers out there is the way to go. Yes, I could’ve stuck with Openfiler by just buying a new RAID controller to save cost initially, but over time I believe I’ll end up paying more money on energy costs by using a regular PC.

Well, I hope this blog post is a little bit helpful for the people trying to build a whitebox ESXi host. Let me know if you have any questions and I’ll try to answer them as soon as I can. Enjoy!

You might also like to read

VMware ESXi Home Lab [2016]

Disclosure

NetworkJutsu.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.

  • Share on Twitter Share on Twitter
  • Share on Facebook Share on Facebook
  • Share on LinkedIn Share on LinkedIn
  • Share on Reddit Share on Reddit
  • Share via Email Share via Email

Footer

WORK WITH US

Schedule a free consultation now!

LET’S TALK

Copyright © 2011–2023 · NetworkJutsu · All Rights Reserved · Privacy Policy · Terms of Use