Get full control over your ESXi Server
May/0814
As you probably know ESX 3i is the embedded version of ESX, mainly run from usb flash disks build-in servers from HP, Dell, IBM and FSC, but you can also install this on your own servers. I really love this version as it is so easy and simple and adds extra (and less) functionality to ESX. The main thing about 3i is that the Red Hat service console has been removed and replaced by a small busybox linux version. To keep this service console tiny, 3i comes with build-in hardware monitoring, something a normal ESX server does not have (wish it did). When connecting to a 3i server with your VC client you can see fan speeds and temperatures in your system and you can of course put alarms on them, in case something breaks.
By default the 3i server gives us no access to this small service console, but there are some down sides to this, as certain things can only be done from the service console
and until VMware builds those missing functionalities into the VC interface you still need to get yourself console access. In the beta versions of 3i there was an option in the advanced settings, but this has been removed in the final releases. VMware has published a knowledge base article that describes how to get console access (this is not officially supported by VMware). See the KB Article. In short you just hit Alt-F1 and then type in ‘unsupported’. The article also explain how to disable this Tech Support Mode, in case you want to.

Of course you do not want to be in the server room all the time and your server might not have some kind of remote access facility, so the first thing you probably want to do is enable SSH access to your server. (Thanks to Lee for this information, and again this is not supported by VMware)
So why do you need console access?
There are multiple reasons why you want to have console access. One of the most common reasons is that you might want to change HBA driver options. To get better performance out of your Qlogic or Emulex HBA most people increase their queue depth (see for instance VMware 100.00 io blog post). To find out what options you can set on your drivers you can use the command vmkload_mod:
vmkload_mod --list vmkload_mod -s /mod/your_driverAnother reason why you definitely still need console access (unfortunately) is if you want to use thin provisioned virtual disks, a new feature in ESX, but for some reason not exposed in the VC interface yet (i think because it is still experimental supported). Any virtual disks created via the normal interface are pre-allocated disks, so a 100GB virtual disk will use 100GB on your VMFS. With thin provisioned disks the actual virtual disk file will start very small and only grow when you will actually need the disk space. Note: this only works for Virtual Disks on iSCSI and FC SAN, not NFS!
With the vmkfstools command you can create new virtual disks as ‘thin’ disks. After you created the disks, you can then use the normal VC interface to add that disk to your VM.
Creating a 100GB thin disk:
vmkfstools -c 100G -d thin /vmfs/volumes/san_vmfs/my_vm/thin_disk.vmdk
If you already have a pre-allocated disk, you can convert it to a thin disk as well. Well it is not really converting, but creating a new copy as thin disk. After you have done that, you will need to remove the old disk from the VM and add the new converted copy.
‘convert’ copy a existing virtual disk to thin format:
vmkfstools -i /vmfs/volumes/san_vmfs/my_vm/large_disk.vmdk -d thin /vmfs/volumes/san_vmfs/my_vm/new_thin_disk.vmdk
Well I am sure there are more reasons why you still want to ‘play’ on the service console, as I am born with a commodore 64, I always want a command prompt
Sorry, the comment form is closed at this time.
No trackbacks yet.
11:03 am on May 26th, 2008
Hi Richard,
I think hardware monitoring is available since ESX 3.5 Update 1, or do you mean something else?
Viktor
3:45 pm on May 27th, 2008
Busybox isn’t really a Service Console replacement, except perhaps from an admin access standpoint. All the virtualization work that the SC used to manage is fully handled by the vmkernel in ESXi. There is no Linux kernel in ESXi at all.
Busybox is a compiled command containing various linux tools (kind of like command.com in MS-DOS). The vmkernel is written to interpret the execution of Busybox so that it thinks it is running in Linux. Think of this as Linux emulation. VMware refers to this by saying that the vmkernel supports “primitives”.
Also, your find is documented by VMware, albeit only officially supported when one has engaged VMware technical support. (I point this out not because I agree with the policy, but so your readers know what they are getting into). http://kb.vmware.com/kb/1003677
I completely agree with your point about the limitations imposed by the headless nature of ESXi (such as thin provisioning). I can’t recommend ESXi under most circumstances at this time. It still has a lot of growing up to do, and the Service Console provides a lot of power to the knowledgeable admin.
2:38 pm on May 28th, 2008
Hello,
I would be hardpressed to think that the vmkernel (which is NOT Linux) has been rewritten just to support BusyBox. The general vmkernel primitive is a VM, or vSwitch, so BusyBox is running within a VM. That VM happens to be booted using a Linux and BusyBox is the environment started. There is no requirement that the full GNU environment be started.
If Busybox is running outside a ‘VM’ then there are a number of security issues that come to mind….
The main busybox.net site contains quite a bit of information on what it will run upon.
Best regards,
Edward
1:34 pm on June 2nd, 2008
[quote]
All the virtualization work that the SC used to manage is fully handled by the vmkernel in ESXi. There is no Linux kernel in ESXi at all. [/quote]
Someone needs to learn what busybox is all about…
http://www.busybox.net/
6:23 pm on June 7th, 2008
Thin disk provisioning is suicide for most customers and yields no benefit at all.
The benefit of VMFS is that it pre-allocates the whole vmdk file and stores that information in vmkernel while the VM is running. That means vmerkenl does not have to allocate new disk blocks when the vm needs it. This is not the case with thin provisioned disks.
I dare anyone to test a high IO SQL server with thin versus thick (default) and see how think provisioned disks outperform.
Also, if users over allocate the vmfs (the only real reason to use thin) and the vmfs volume fills up because the user does not monitor the filesystem then you can kiss some of your VMs goodbye.
9:32 pm on June 16th, 2008
I am able to do all of this to get to the hidden SC, however, I can’t seem to remote in and I still can’t create any VM’s. Anyone have a good guideline and step to step on how to do this? For something simple as the ESXi is marketed, I think I am stuck.
Thanks to anyone that can help.
2:00 pm on June 18th, 2008
NoThinDisk wrote that he dare anyone run a High IO SQL Test. – My question is how much does that matter over a iSCSI or FC SAN where you have a large amount of cache? how fast can VMWare grab inodes on a system like that?
Maybe if you are writing a large amount of data, but you can have a lot of SQL IO without writing a lot of data to disk, right?
Would you say thin disk are ok for dev, if not dev and qa systems.
Thanks,
Mark
8:19 am on July 5th, 2008
I want to thank you about showing me how to enable the SSH server inside ESX3i. This will give me the possibility to access the system remotely and savely. Furthermore I can execute commands like vmkfstools from everywhere, beeing able to clone my disk from another system having cron enabled
Indeed you’re right saying we need a console. I am working with Unix from the 80’s and cannot live without a command shell
6:59 pm on July 15th, 2008
Excellent information! Works perfectly. Now I will just have to look more into this BusyBox thing…
/Henrik
9:53 pm on July 31st, 2008
Good news that there is still a console access. Does anyone know if they used an existing mini-distribution like DSL or whether they rolled their own? Also, which ones of esx-specific commands (e.g. esxcfg-info) exist in addition to the usual busybox commands?
12:06 am on August 13th, 2008
Thank you for the article… I had no idea that I could type ‘unsupported’ at the console to gain shell access.
VMware has apparently taken their distro from the basis of Red Hat Linux (so an Engineer from VMware told me). They use BusyBox mostly for the size advantages, mostly especially since ESXi can be in embedded devices (since it’s much smaller than separately compiling a bunch of these fully-featured utilities).