www.run-virtual.com Everything you always wanted to know about Virtualization

I’m still laughing…

Please advice when your company will disclose this post and will include some blocking mechanism on upcoming ESX Server 3.0 new build (I really hope this won’t happen but you know…).

Alessandro Perilli, CISSP, MVP
http://www.alessandroperilli.com

Blogging about IT Security on http://www.securityzero.com
Blogging about Virtualization on http://www.virtualization.info

I too would like to do this. I already have a server with PCI-X.

Please tell us the exact changes you made to the /etc/vmware/vmware-
device.map file.

Thanks

Is it possible to make a list of servers with SATA disks that could be candidates for ESX server to install on them?? by instance, Sun servers out of v20z and v40z?

Best regards

The change required is very small, run lspci and get the id for the 300-8x card. beleve that it is 0×0409

make a backup copy of vmware-device.map

copy the line where the id is 0×0408 and change the copy to 0×0409 (could be another number, does not have the card in the computer right now) make sure that the module is megaraid2

reboot the esx server.

after the reboot, go to the web gui and change the startup profile, to share the megaraid card.

reboot the server

after the second reboot you can create vmfs partitions.

I hope that this instruction taken only from memory is to some help.

perhaps someone could make a complete howto

Wouldn’t a virtual iSCSI solution like the one from http://www.rocketdivision.com be a much cheaper alternative to this?
Then you could just share the storage of a connected server as a virtual SAN.

I think it’s also possible to put that modification in: /etc/vmware/vmkmodules.conf.local so your modifications won’t be lost after an upgrade.

I’m very, very curious in how well this performs… Could you do a test anykey, agains a scsi disk or sata disk?

I don’t understand the logic here.

Did Richard (the author) used a IDE to install ESX first, then modify devices.map, then bring up the 300-8X for ESX to see, then install VM on to it?

As you can’t make this change during installation, so you won’t even able to install ESX directly to 300-8x at the first place.

There is currently a SCSI requirement for the VMFS volumes but not the ESX load itself which can go on a local IDE drive.
With ESX 3.0 you can use ANY kind of disk (PATA, SATA, SCSI etc) behind NFS or iSCSI, just plan for the sustainable I/O loads accordingly.

Hi,

I think it is possible to through in info in the device.map-file for any raid/ide/scsi/sata and the kernel will most likely load the driver and let it have VMFS. Just run lspci once you installed Vmware ESX. Then put in the right values in devicemap-file and reboot. After reboot maybe the driver has loaded for you IDE-disks. To load a driver for Intel 845/ICH4 just copy pci-ide.o or appropiate driver to vmware-modules folder and edit vmware-devices.map. Maybe it works. It’s untested AFAIK ,=)

Hi

Im have installed a RockRAID 1820a SATA RAID card onto a server built to run ESX server. When I install ESX server it states my RAID card is not supported and supplies me with the following information:

Bus – 04
Slot – 9
Func – 0
VenID – 11ab
DevID – 5081
SubVenID – 0000
Class – SCSI Storage Controller

When I run the lspci – n command I receive the following info about my card:

04:01.0 Class 0100:11ab;5081

Could someone please tell me the exact syntax this information needs to be entered in the vmware-devices.map file. Do I need to install the driver into ESX server? If so what is the path?

I have found driver for Red Hat 7, are these correct?

Thanks in advance for helping a linux newbie

SATA Disks : Can the author advise what make of SATA disk he used ?

I have a Western Digital that the Lsi Controller has picked up but it can not see my Hitachi 400 GB drive.

Above method works for ESX server 3.0 using LSI Megaraid 300-8XLP (same PCI id as 300-8X):
1. install on IDE disk
2. place line (changing 0×0408 to 0×0409 as described above) in /etc/vmware/vmware-devices.map.local
3. update PCI device list for boot purposes: esxcfg-boot -p
4. shutdown ESX server
5. copy the contents of the IDE disk to a LUN on the raid controller. I used Knoppix to copy the first 32GB: dd if=/dev/hda of=/dev/sda bs=1M count=32768
6. reload linux modules for raid adapter (”megaraid” and its siblings) to cause kernel to detect the partitions, mount the ESX /boot partition (it’s ext3)
6. Edit /grub/device.map, change “(hd0) /dev/hda” to “(hd0) /dev/sda” (or sd[a-z], depending on which LUN on the raid adapter you used)
7. set the raid controller to boot and server BIOS the LUN, remove the IDE drive
8. boot ESX server from raid array
9. ask vmware to provide real support

Displays as “Model: Unknown” in VI client, probably because there is no corresponding entry in pci.xml (anyone know if there is a “pci.xml.local”?)

Works for me on 3.0.0 and 3.0.1, no difference in configuration.

On the hardware side the 300-8XLP has proprietary SATA octopus cables, one of mine needs some fiddling to work properly. LSI will replace the cable, but needs the whole card, package, and cable kit returned before they’ll replace. Outstanding.

Never ran into that issue. Some potential questions

0. Sure it’s a “supported” LSI RAID controller?
1. Have you verified things work with regular Linux using the expected modules?
2. Output of “lspci” and “lspci -n” from service console running from IDE disk? Is 0×0409 displayed as expected?
3. What is the entire line in /etc/vmware/vmware-devices.map (or vmware-devices.map.local)?

If you’re planning to boot from SATA controller,
0. Did you run “esxcfg-boot -p” before transferring contents of IDE disk?
1. Did you transfer contents of IDE disk to the right LUN and update grub?

Sounds like you’ve done the same things in configuration. Maybe a hardware situation? Chipset, PCI, or interrupt situation?

Some additional info, other potential approaches:

1. Am booting from a LUN on the raid device, so the megaraid.o module may be loading earlier than it does for you. Not sure when vmkernel masks PCI devices for the service console, but this may be useful info

2. Results of esxcfg-boot -q vmkmod from running service console:
vmkapimod vmkapimod
vmklinux linux
tg3.o nic
megaraid2.o scsi

3. Not really familiar with service console and vmkernel handling of PCI devices, but the “cpci” option could be a potential sticking point if not set correctly. Mine is “cpci=2:1;3:;”. Note that the LSI 300-8xlp is in 3.

lspci excerpt is:
02:01.0 PCI bridge: Intel Corporation: Unknown device 0335 (rev 0a)
02:03.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 10)
02:03.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 10)
03:0e.0 RAID bus controller: LSI Logic / Symbios Logic: Unknown device 0409 (rev 0a)

lspci -n excerpt is:
02:01.0 Class 0604: 8086:0335 (rev 0a)
02:03.0 Class 0200: 14e4:1648 (rev 10)
02:03.1 Class 0200: 14e4:1648 (rev 10)
03:0e.0 Class 0104: 1000:0409 (rev 0a)

4. 8x vs 8xlp models do use the same firmware and have the same PCI ids, but it could be a difference in models

Hope some of this is useful. Good luck!

I’m using the LSI Logic MegaRAID 300-8x SATA II card and have managed to install ESX 3, providing the LSI Logic MegaRAID driver when prompted for a storage device so it thinks for all intents and purposes that I’m using SCSI s. Everything seems fine until I reboot, at which point it seems to boot into ESX but then I just get a # prompt (not the login prompt) with a few lines preceeding it, one of which says “Busy Box”. Don’t know 100% what that is (I read somewhere it’s a compact administrative/troubleshooting kernel of some sort), nor do I know what to do to resolve the issue. Tried re-installing with various on-board hardware disabled with no better luck.

Does someone have any idea of what problem I’ve got going on? Solution?

I wanted to report back that I got the MegaRAID card to load (!) and the ESX server to boot. I followed the following directions I compiled from all of the responses on this thread:

1) First I started up into the Service Console (third choice on the startup menu)
2) Ran the lspci command and sent the output to a file so I could read through it (you can also pipe it to something. Upon review of the file I was able to determine which device coincided with the LSI Logic MegaRAID controller. In my case, it was 0409 located at 03.0e.0.
3) Then I ran “lspci -n” (again sending the output to a file for review, although you can certainly pipe it to a viewer of some sort) as advised and was able to collect the rest of the required information once I found the card listed in the file based on the 03:0e:0 and the 0409 info.
4) I then modified the file /etc/vmware/vmware-devices.map, adding a line with the following, using the info I collected from the lspci -n command. I added the line immediately proceeding three other lines referring to the LSI Logic MegaRAID controller (that’s where I got the last two comma-delimited entries on the line):
device,0×1000,0×0409,scsi,LSI Logic MegaRAID,megaraid2.o
5) Typed the command esxcfg-boot -p
6) Typed the command reboot

At this point, the server restarted and came up properly. I still cannot access the server via the VI client, but I’ll work on that one next. I think it has to do with some configuration related to the NIC, but I wanted everyone know at least what I did to get things working with the LSI Logic MegaRAID 300-8x card.

Followed roughly the instructions above, have a scsi drive with esx installed and running.. Want to use sata array as the main store for vm’s..

Created a logical drive in megaraid bios

Installed lsi megaraid sata2 8xlp, added the following line to vmware-device.map:
device,0×1000,0×0409,scsi,LSI Logic MegaRAID ,megaraid2.o

Kernal loads ok

Saved and restarted srv, can now see the SATA array in the startup profile

What do i need to do to assign this as a vmfs or to be able to use this space?

So close but no cigar!!

Are you using a ASUS P5WDG2 WS Professional motherboard with your 300-8x? If so which BIOS version and what are your settings?

Hi,
I’ve setup a esx 3 server with sas discs for the vm’s. This seems to work. Now I’m trying to add a rocketraid 2220 card with four sata disks (RAID5) as storagedisks for the vm’s.

Is it possible to make this work??

By the way – I have tried installing and importing VMs – it all works pretty smoothly…

Hi All

I was running a ESX 3.0.1 server with a LSI logic MegaRaid 300-8x.
In the last 2 day’s i had a lot of troubles because:

1) i updated the firmware of the lsi logic 300-8x to 815C; with the ESX 3.0.1 the VMFS wasn’t available anymore … so I downgraded again
2) after that i was updating to ESX 3.0.2 and made the changes to the /etc/vmware/vmware-device.map file; Result is a purple screen

So I guess I have to install the ESX 3.0.1 again. Or does anybody’s got a solution?

Thanx and Regards

Truemmerli

Hi all

The upgrade to ESX 3.0.2 is running now. It was because I had a copy of the ESX installation on the 300-8x array. So, with the update to ESX 3.0.2 some new support is present. Probably we can install the ESX System on the 300-8x array within the setup process…. would be nice ….

bye

Truemmerli

Hey!

I have a asrock mainboard and havin trouble with ESX 3.01, ist says kernel panic, it worked with an MSI board. is esx sensitive about the brand of the mainboard? all other hw is the same as when i had it runnin on the MSI board. i have upgraded bios.

Great article!

I have a SATA Raid controller (Intel SRCS28X) which is basically
the same as the LSI Logic MegaRAID 300-8x. Apart from Intel rebranding
this card, are there any other issues I can walk into?

If someone can confirm that the Intel SRCS28X will work on ESX, I’m
a happy man!

Thank you in advance!

I can confirm follows:

My Tyan s5360D server with LSI Megaraid 320-1 with SCSI RAID was just upgraded with Intel SRCS28X and two SATA II Hitachi Deskstars.

All seems working smoothly. And I have dual boot option: one from Intel SATA RAID and one from LSI 320-1.

I was just followed what Stan Levine says, but after I have installed ESX 3.0.2 on Intel SATA RAID and reboot I choose boot with console option to gain acces to normal shell.

Choose Intel if you wish – it works the same way

ESX Server 3.0.2 I got the megaraid 300×8 sata card to work but can’t see the Storage adapters in the VM console to add host. I tried the command below but come up with errors. Can somebody help me out on this one thank you.

7. I was not able to format the disk in the VC client, so did this from the service console:
7.1. run: vmkpcidivy -q vmhba_devs -> this will give you a linux device pointer back of the HDD attached to the megaraid, in my case /dev/sda
7.2. run fdisk /dev/sda (or what ever you got back). In fdisk delete any existing partitions and create a new one with the type “fb” (which is vmfs)
7.3. run the vmkfstools -C vmfs3 vmhba? ?:? command to format the vmfs on the LUN. where ? are the lun, target, id and partition number, you will find the in the VC interface under Storage adapters.

I’ve been through about 20 reboots and still can’t get the MegaRAID SATA 300-8x working with ESX 3.0.2 Update 1.

I’ve got a SuperMicro H8DME-2 motherboard with a Opteron 2347HE (quad core) and 4GB of RAM. I’ve tried the 300-8x in several different slots, and tried pretty much every combination of BIOS settings (enable/disable ACPI 1.0/2.0/3.0, enable/disable Plug and Play OS, etc.).

The error seems to be related vmkernel having a problem with setting the interrupt for the card…I have errors like the following (in /var/log/vmkernel):

invalid IRQ 0×0

I also tried the following from a “Service Console only” boot, and it resulted in a kernel panic:
modprobe megaraid2

I installed CentOS 3.9 on this same exact hardware with no problem…the card and logical drive were seen and I was able to install to it.

Any ideas?

OK, here’s my final word on this, now that it is working for me. These are instructions on how to install ESX onto an LSI 300-8x with drives attached only to the 300-8x:

0. Create a logical drive on the 300-8x.
1. Open the ESX install ISO in your favorite ISO editor. I use UltraISO.
2. Extract /isolinux/initrd.img from the ISO.
3. Rename initrd.img to initrd.img.gz.
4. gunzip initrd.img.gz
5. Open the resulting initrd.img in a hex editor and search for the string “0408″. Replace both instances of it with “0409″ and save the edited file.
6. gzip initrd.img
7. Rename initrd.img.gz to initrd.img.
8. Replace /isolinux/initrd.img in the ISO with the changed version you just created, save the ISO, and burn it to CD.
9. Boot from the burned CD and install ESX as normal…any logical drives on the Megaraid 300-8x will be visible during install. But, skip the “check the installation media” function…it will fail (because of the edits we have made).
10. While the packages are installing, hit Ctrl+Alt+F2 to get to the shell prompt. You can wait until the install hits about 50% if you want.
11. cd /mnt/sysimage/etc/vmware/pciid
12. tail -F megaraid2.xml
13. When you see some output from the tail command, hit Ctrl+C, then edit the file, typing exactly the following:
vi megaraid2.xml
:%s/id=”0408″/id=”0409″/g
:wq
14. Hit Alt+F7 to return to the graphical install screen. The last thing the install does is run “esxcfg-boot -p -b -g”, which uses the changed megaraid2.xml and creates an initrd that will boot your system.
15. Click “Finish”, remove the install CD, and boot directly into your 300-8x.

No need to use a hex editor to edit the file in the initrd. Just gunzip it and mount it as a loopback.

mount -o loop initrd.img /mnt

Then edit the correct file and unmount. The install went fine for me, but I am getting horrible write speeds to the array. Does anyone else have slow speeds. I am talking about taking five minutes to run the following command:

dd if=/dev/zero of=testfile bs=1M count=500

I am running a raid 5 with 4 320gb Seagate SATA drives on a 300-8x under ESX 3.0.5. Any help would be appreciated.

Well, I was able to get it to work on my system with ESX 3.5 Abit FP-IN9 Fatal1ty board works wonderfully for SATA. I currently have about 2TB of storage on my board using the native controller on the board. This of course is for home use and not production. But the board is rock solid for me. I have had the machine running with no issues for me.

I have had no issues at all in setting it up. Works right out of the box.

Guys, I have good news for you on the ESX 3.5 SATA support. You only need less than $20 to get a good working SATA storage solution for ESX 3.5. I spent roughly $400 so far until finding this out, so I hope this would be beneficial for you out there that need a cheap working solution. (YES, this works for the VMFS datastore, I uses standard PATA to install ESX server.)

1. The cheapest solution is to go with a SATA card that uses Silicon Image chipset (almost ANY Sil chip should work, you can look at the /etc/vmware/pciid/sata_sil.xml file for specifics, but it seems that it works for ALL Silicon Image chipset that is out there, except for the PCI Express one, which is not in the list, but I think it should work if you follow the instructions as described in this thread). You can get the Rosewill RC-201 PCI 2-port SATA at newegg.com for $11 each, or go with a fancy one with 4-port such as the Rosewill Silicon Image RC-209-EX PCI, which as 2 eSATA and 4 internal SATA (but only 4 can use at one time) for $30 each. I bought a Vantec 2-port SATA card at Frys for $30, which has the Sil3512, and this has been working flawlessly for me so far. ESX 3.5 will automatically detect it so it’s a piece of cake to set up.

2. You can also go for Promise Technology cards (which is a little more expensive, but still much more affordable than LSI). You can look at the /etc/vmware/pciid/sata_promise.xml for the supported ones.

I had the Intel 965 + ICH8 on the motherboard, which has built-in SATA ports, but get me into hell and I finally quit trying. It seems that the ata_piix.xml shows that ESX 3.5 has support for almost all other ICH series chip, except for the ICH8 which is used with 965 chipset. I somebody can get this to work, please help us here since it would be nice to use the on-board devices. The 965 chipset happens to be quite popular.

Has any tried with the Intel chipset that has ICH7 or ICH9? Please let us know.

Many thanks for this article since it has been very helpful for me in search of an affordable solution to try out ESX 3.5.

I’m assuming the 4x cards also work ?
the 8x are a bit pricey

myv999 – Thanks for the reply. I have ordered a UGT-ST300 card as this has the same chipset. I’ve read that you may need multiple drives in a RAID configuration to enable ESX to see them as SCSI. Have you found this or have you just used a single drive with no RAID using your controller?

Hi Tim,
There is no need for RAID, a single drive would work just fine. I just use this as a development machine with a Maxtor 500G SATA drive bought at Fry’s on sale for $60 and it’s been working nicely. I was using the VMW Workstation and there is no comparison in terms of reliability and performance with ESX. I had a power fail in my area, and lost ALL my data on VMWare Workstation. That was a painful experience. Will evaluate ESX if that is a problem or not.

I did not actually try the RAID configuration since this was only for development, so just need to be cheap and working.

Can anyone check the sata_sil.xml file if the FastTrak TX2300 card is supported for ESX 3.5 ?

thanks in advance.

I was not able to get Rosewill Silicon Image RC-209-EX PCI working. I tried both raid and no raid configuration. It installs fine, but after rebooting, the system just hangs before displaying the boot menu. Was anyone successful installing ESX 3.5 with this SATA controller?

Thanks,

And the results for the fast trak TX2300; I works just out of the box, no sweat at all. I put the card in my Dell XPS 420 connected the sata HD, put the ESX3.5 CD in the CDROM boot it and it just works, wauw! a small notice that you must also have a NIC that is supported by ESX (see vmware website) total cost 46 euro.

GREAT POST! Got a question though.

I use an ABIT-IN9-32xMax mother board with an integrated Nvidia Raid Controller. If install ESX 3.5 on my 70 GB SCSI Drive (using an Adaptec 29160) I can the go in an edit the sata_nv xml file and change the entry at th ebottom labeled 037e to 037f. Run the commands you suggested, and wala! I can see my drives and use them for VM’s.

If on the other hand I run the ESX 3.5 installed, during setup it sees and writes to my sata drives, with no errors. At the end of the install I reboot and get NOTHING. Which makes some sort of sense because I haven’t run your commands yet, but not a lot of sense because it sees the drives and writes the whole install to them.

i tried opening a command prompt during the install and before the install asked me to eject the disk and restart I attempted your steps, but I get perl errors, so i am assuming the enviroment is not really setup correctly.

Should i give up trying to do this and by a SIL controller, or is there some magical way to chroot (which i know of but not how to use) to modify the install after I set it up on a sata drive using some boot cd or something?

Also what is the latest consensus for compatible controllers under $50?

Thaks alot.

Ron

Hello All!!!

I have successfully got ESX Server 3.5 working on a Intel SRCS28X Sata 2 Raid Controller. If anyone would like to know how I did it, I am happy to post a step by step set of instructions for you.

My Server Specs are:

Intel SC5200LX Chassis (Hotswap Everything/3 X PSUs)
Intel SE7501CW2 Motherboard
2 X Intel Xeon 3.06GHZ CPUs
8GB DDR ECC RAM
Intel SRCS28X Raid Card
8 X Seagate 400GB Sata 2 Hard Drives in Raid 5 Configuration

Cheers,

Bruce…

Hello Bruce,

Can you write a brief about how to you do to work the Intel SRCS28X Raid Controller?

Thanks,

After trying a fresh install of ESX 3.5 (added the megaraid2 driver) without changing the iso, i got the famous ‘Can’t load root’ error.
Did a fresh install of ESX 3.01 and followed the instructions given on these page to make the MegaRAID SATA 300-8xlp usable within ESX.
Then performed an upgrade to ESX 3.5 (had to add the megaraid2 driver and used the iso of 20080220) and followed the new instructions for ESX 3.5.
ESX 3.5 is working fine now.
Used systemboard is a Tyan Thunder h2000M (S3992).

Have someone an Idea how can i add a Network driver for
Realtek 8111B chip.

I can’t continue the Installation because there were no Network devices found.

Hey hey,

Wrecking my head trying to get ESX 3.5 installed on the new hardware I just bought.

Am familiar with editing files and booting into linux and what have you just stuck on what I need to do to get this particular On-board SATA controller to work.

I bought the Quad Core 6600 for Virtualisation (To see if there is a benifit?) and the mb I am using is the Gigabyte GA-73PVM-S2H,nForce-630i+GF7100.

I want to use the 2 x SATA HDD in RAID 1 but it doesn’t allow to be picked up. I tried to install on one drive but it boot and says failed to mount root once installed.

Reading above, I realise I can get a seperate controller card (it would have to be low-profile) but I would prefer to use the onboard.

Also, again, if I have to using an IDE for boot could be an option but I would still prefer to use the RAID 1 to store the VMFS if possible?

Please help if you can as I am at whits end.

I just wanted to share my recent success with installing ESX 3.5 on LSI Megaraid 300-8X; thanks to many of the posts on this list including Jeff Rife’s.

The only thing I wanted to add to his, was that editing megaraid2.xml file is time sensitive and must be completed before Esx setup runs “esxcfg-boot -p -b -g”

This only came about as a problem for me because ESX 3.5 would not allow me to use the CD for a source (no clue why) so I used a FTP server instead (with the modifications as mentioned in Jeff’s posting.)

However, with the FTP the install is lighting fast and we had to set up speed limits on the FTP server to give us enough time to make the modifications.

After that it was all good! Tonight I am going to be installing ESX 3.5 on a LSI-3041E-R card.

Can you tell me (or guide me) to find out how to make intel’s ICHx RAID to work with ESX?

Has anyone tried to get any of the Highpoint RocketRAID cards to work? I have a RR3320 (PCI-E, 8 Port SATA II controller) on a Dell Poweredge 2950?

using the posts above I have found the Unknown device ID (0103), but I don’t know which file to modify.

Thanks,

-B