Tools and Utilities

Virtualization Tools and Utilities

Easily copy an ESXi VM to your desktop

Hi,

I thought I would share with you a simple way to copy a VM from your ESXi servers to your desktop. In my case, my local NAS (qnap) device had a diskfailure, so it is rebuilding its disks for the next 2 days, which slows down IO for my VMs (mainly for my work VM), so I temporary wanted to copy the VM to my local PC and run it in workstation.

Using the build-in download option in the vsphere client sucks. It was super slow. I tried using sftp, again slow (bit faster then vsphere client). My first tought was to google “enable ftp server on esxi”, which I did, and did find ways to get an ftp server installed on my ESXi boxes, but while I opened the ESXi firewall, it still had problems opening a data connection. So after 15 minutes fussing with that I quit that road.

In the forums about FTP on ESXi I came across posts to use Veeam FastSCP, so let’s give that a try. Turns out that the software does not exists anymore and is integrated in a 500mb install of Veeam Backup. I have fast internet, so sure I can download then, but then during the install.. “do you want to install local sql server or use existing??” Hello, I am just trying to copy a single VM, I do not want to screw up my local machine with all that stuff, so cancelled that as well.

Mmm.. what else to do… went googling again and then stumbled on a Freeware bit of software called FreeNFS (http://freenfs.sourceforge.net/) It seems very recent as the note from the developer is from August 14th 2012. It is the most simplest plain NFS server software for windows. its a single executable, click on it, and it runs. done :-)

So now it was easy. As my local machine suddenly was an NFS server, I add the NFS datastore to my servers and just used a clone VM operation, to thin disk, to my local datastore :-)

Thank you Lawrence for making this great Free NFS software! Works great with ESXi (version 5).

FreeNFS – http://freenfs.sourceforge.net/

sql_sample

VMware View – Getting data out of the Events Database

Well I started work in my new version of vAudit, making more functionality and most important support for View 5. In case you do not know what vAudit is, it is mainly a tool to understand who and when is using your View environment. This can help you see how well the adoption of your VDI systems is going, when not to plan maintenance, etc.

In the old version of vAudit I used WMI to query the event logs of the brokers to see who was logging in and out. Since View 4 the products comes with the option to store all events in an events database (Microsoft SQL or Oracle). So for this new version this is what I want to use. Unfortunately VMware’s View engineers are not easily storing the event data (and this is an understatement!). So it took me a while to even get some basic information out of the system. As my vAudit is not ready for release yet, I thought I would at least share some SQL statements with you, in case you want to start cracking :-)

Checking daily max concurrent users.
Well this is the easiest (and the only easy thing) to do. As the broker service daily writes an event 5 minutes to midnight in the events database on that days max concurrent user. You can easily get this using a query like this: (MS SQL)

select Count, Time from(select top 30 dbo.view_event_data_historical.IntValue as 'Count', dbo.view_event_historical.Time as 'Time' from dbo.view_event_historical,dbo.view_event_data_historical where dbo.view_event_historical.EventID = dbo.view_event_data_historical.EventID and  dbo.view_event_data_historical.Name = 'UserCount' and dbo.view_event_historical.EventType='BROKER_DAILY_MAX_USERS' order by dbo.view_event_historical.Time DESC) A Order by Time

This might seem like an extensive way of doing this, but it will allow you to control the last so many days to query. Just change the ‘top 30′ to any amount of days.

With this data you can chart a nice daily max concurrent user overview. See my screenshot of the new vAudit release. (I did alter the data in the database to make it look like I have a lot of sessions).

NOTE: also on the name of the tables. When you install view, it asks for a table prefix. In my case I supplied “view” as prefix. So make sure to check your prefix and modify the table names based on that.

 

Session overview

If you want to make more sense of the sessions used by your users, it becomes more of a challenge. Each time a users logins to the broker an events is written. The same for when they session is broken/loggedout. In the event table these events are not easily connected :-( so you would have to query for each the event_data table as well to get the broker session ID and match them all up. The super irritating thing is that the time stamp is NOT in the event_data table but just in the events table. This would else have made it into a very easy query. So after a lot of googling, cursing and wishing I wan an SQL expert, here is what I came up with.

drop table #sessions
drop table #logoffs
drop table #users

create table #sessions (SessionID varChar(32), StartSession datetime, EventID int)

insert into #sessions (SessionID, StartSession, EventID) select view_event_data_historical.StrValue, view_event_historical.Time, view_event_data_historical.EventID  from view_event_historical, view_event_data_historical  where view_event_historical.EventID = view_event_data_historical.EventID and view_event_historical.EventType = 'BROKER_USERLOGGEDIN' and view_event_data_historical.Name = 'BrokerSessionId'

create table #users (EventID int, username VarChar(512))

insert into #users (EventID, username) select view_event_data_historical.EventID, view_event_data_historical.StrValue from view_event_data_historical, view_event_historical  where view_event_historical.EventID = view_event_data_historical.EventID and view_event_historical.EventType = 'BROKER_USERLOGGEDIN' and view_event_data_historical.Name = 'UserDisplayName'

create table #logoffs (SessionID varChar(32), EventID int, EndSession datetime)

insert into #logoffs (SessionID, EventID, EndSession)select view_event_data_historical.StrValue, view_event_historical.EventID, view_event_historical.Time from view_event_historical, view_event_data_historical  where view_event_historical.EventID = view_event_data_historical.EventID and view_event_historical.EventType = 'BROKER_USERLOGGEDOUT' and view_event_data_historical.Name = 'BrokerSessionId'  

 select #sessions.SessionID, #sessions.StartSession, #logoffs.EndSession, #users.username from #logoffs, #users, #sessions where #logoffs.SessionID = #sessions.SessionID and #users.EventID = #sessions.EventID

The query uses some temporary tables to help match the session ID’s, get the Start time from one event, the end time of an other event and get the displayname of the user. See the picture of my result.

Next step would be to use this data to make daily charts on when most uses are logged in and out, users stats like average session time, etc. So more to come :-)

Just to extra clarify what this query does, it check sessions to the Broker! not to the actual VMs. You can do this as well, but even more complicated (as those do not have unique IDs like BrokerSessionID). The the list shows people logging into the broker, but you will NOT know if they started one, two or more virtual machine sessions.

 

If you have any SQL work for View you want to share, please do! If you can optimize my lame-ass sql queries, please do :-)

 

I will hopefully soon be posting a beta of the next vAudit that does all this and translates it into nice charts.

 

 

 

Scary! Is your VMware server being hacked???

2 Weeks ago I attended the Blackhat security event in Barcelona and got me thinking about security around the VMware platform. At the show there was even a session about vulnerabilities in virtualized environments. Happy to hear there where no major leaks in the ESX layer, it still got me thinking. What is someone right now is trying to hack into my ESX hosts or my vCenter environment, would I know??

During blackhat I also saw a tool demonstrated that I never had heard of. It is called OSSEC and is made by Trend, but is free to use by anyone. OSSEC is designed to collect centrally all the logs from servers, including ESX, ESXi and Windows (vCenter) servers and analyze the entries to raise alerts when it detects something wrong. For instance if someone was not successful in logging in. It can even increase the alert when it notices if multiple failed login attempts happen in a short time span; this could indicate that a bruteforce attack is taking place.

I found that OSSEC worked quite simple in defining rules and was even more surprised that it came with default build-in VMware rules. unfortunately designed I think for 4.0 as I had to make some small changes for my 4.1 environment to accomodate VMware’s updated log syntax.

OSSEC’s success depends on the quality of the rule set. So I am really hoping more people with real world VMware environments have a look at this and share their experiences against what would make good rules. To make this a bit easier, I have written a simple ‘manual’ on how to install ossec and configure ESXi host and vCenter servers to remote log against the OSSEC server.  You can find the instructions here: http://www.run-virtual.com/?page_id=690

You might wonder (like I did), who cares? Should the VMware management environment not be on its own isolated network? Well I found out that in the real world this is not the case. Last week I did a workshop around security with 15 customers and I think all had an environment that any desktop in their company could access their vCenter and/or ESX hosts. So if any of these clients would get infected by some kid of VMware Virus, it could attack!

After hearing that so many people had their VMware environment ‘so’ exposed, I wondered. Are there any ESX / vCenter servers directly attached to the internet? So the hacker in me decided to spend an evening building a ‘vmware scanner’. I made it fully multi threading (about 800 threads per program) and spinned up 5 virtual machines running this scanner. This allowed me to scan roughly 4000 IP addresses every 30 seconds (max winsocket timeout)… The scanner was build using the VMware APIs, so it could not only detect if an IP was a VMware server, but also what version and build number. Yes, requesting this information can be done ‘anonymously’ with no special permissions.

… 24 hours after scanning and suffering from having bad internet at home because all the scanner bots where eating up my outgoing bandwidth I was completely shocked. Each of my scanner bot had long lists of ESX, ESXi and even vCenter server IP addresses, including some very old versions (which I wonder if they have them fully patched).

Well I do hope that none of your VMware boxes are directly internet attached, but the real world has shown that most people do have their VMware servers attached to their entire internal network. I understand the practical reason behind this, but you should then also make sure you have the right measurements in place to detect any attacks.

I would suggest, tryout OSSEC, let me know what you think and let’s share some good rule sets, so we all can detect the instant something ‘strange’ is going on, from security attacks to hardware failures.

The OSSEC/VMware install manual: http://www.run-virtual.com/?page_id=690

 

Free Partition Manager for Virtual Machines

Ever need to change the size of your Virtual Machines? Well, changing the size is fairly easy, but what about the data and partitions inside your Virtual Harddisks? Paragon has made their “partion manager 10 for Virtual Machines” available for free! (well at least the first 5000 copies). Besides changing partitions, it can also do other handy tricks like backups and merging partitions. The product supports the ‘new’ GPT partitions that operating systems like Windows 7 and OSX uses.

Get here your free copy :-)
Partition Manager 10 for Virtual Machines product page

2010 starts with a nice new Virtual gadget

Happy New Year to you all! CES just started and IOMEGA, now like VMware an EMC company just released a new product called v.Clone. It’s simple, take any IOMAGE external harddisk and you can clone your PC and later on keep it insync (2way), take the external disk anywhere else and with the embedded vmware technology (a custom version of VMware Player) you can run your PC on anyone’s system! Wow nice.

There are some catches, the V.clone software will only work with IOMEGA harddrives and you will need local admin rights on the computer where you want to run the virtual machine. Win7, Vista and XP are supported.

According to IOMEGA, v.Clone is the first product designed for individuals to move seamlessly back and forth between a physical environment (home or office PC) and a virtual environment (Iomega Hard Drive connected to any PC). This allows the user to have a complete, usable virtual copy of their physical PC with them at all times. v.Clone keeps your data in sync between your primary and secondary computer so your virtual environment is ready to go at a moment’s notice.

No pricing information yet, but IOMEGA claims it will be coming in January.

http://ww2.iomega-web.com/vclone/

vAudit 1.5 released

I just finished an updated version of vAudit. In case you do not know what vAudit is, it is an utility that can connect to your VMware Connection Broker and report about user activity. This new version includes 2 major updates; 1. the program can be resized 2. the information can be exported to a comma seperated file.

You can find vAudit 1.5 on the download page

UPDATED: vAudit 1.00

Thanks to Jeff and Sudharsan for your feedback! I just made a new version available of vAudit with 3 new improvements:

- vAudit now also checks for login failures, so you can detect if people are trying to hack into your system
- You can now resize the username column, so you can actually read the username if you have long domain names
- When you MouseOver a session, it will display the machine name and time information

In the next release, I will try to add information about your top users, make an export function and at some point will give your a graph with daily concurrent usage.

You can download vAudit 1.00 here

powered by Virtacore