I/O Benchmarks, VMware, RAIDs, & SATA vs SAS

The need for more space and less power consumption in my home lab has arisen.  I pulled out a RAID 5 array composed of four 15k 146gb SAS drives in favor of a RAID1 array composed of two Western Digital 2tb Red NAS drives.  This server would now have a 256gb SSD, a RAID1 WD 1tb Enterprise drives, and a RAID1 2tb Red drives.  The hope is that the slightly faster Enterprise SATA drives and the decent RAID card would be sufficient, and the NAS drives should be fine for data.

I ran some benchmarks; this time I ran them with a larger data set, as the 512mb cache on the RAID card throws off benchmarks.  We need to test the drive arrays, not the RAID card cache.

four drive RAID5 146gb 15k SAS

two drive RAID1 1tb WD Entperise SATA

two drive RAID1 1tb WD Red NAS SATA

I am really surprised that WD Red NAS drives out performed the WD Enterprise (WD1003FBYZ) drives.  Both are 64gb cache, the Reds are 5400~7200 rpm (Intelipower) and the Ent. are 7200rpm.

I/O Benchmarks, VMware, SSD's, & 6gbps SATA

I have been a big fan of sticking a SSD drive into VMware hosts for a long time.  Using Host Cache (where the ESXi host will use the SSD for it's swap file location), and redirecting the VM swap file to also be on SSD.  In my home lab I upgraded from a Corsair  Force90gb SDD purchased back in 2012 to a Crucial M4 256gb.  A bit more room, and I figured a 3+ year newer drive might have some performance gains......turns out not really.

I also picked up a cheap SATA 6gbps controller hoping that might help pick up some more speed...turns out ..meh...not really.  ALSO, most SATA controllers are not supported by ESXi v5.5 and newer.  I had to take advantage of a community hack to get it to work:

VMware local disks needs WriteBack Cache

When using VMware servers and local hard drives, make sure one uses a battery backed cache card.

What an operating system does when it wants to write to a drive, is it waits for confirmation from the drive/controller to make sure the data is written before it sends more data (write-through).  This is VERY SLOW.  Windows gets by this by doing some creative caching that eats up system RAM; where as VMware, relies solely on the hardware.  The alternative is write-back, where the drive write confirmations are done by the controller, the data is cached while waiting for the disk to actually write it, and the OS continue to sends more data.  In a way it is kinda sorta like TCP vs. UDP.

Just to test this...I have an HP Proliant ML310 with four 250gb 7200rpm SATA drives in a RAID 10 configuration; the RAID controller is a SmartArray P212.  Vmware v5.5 is installed.  The test VM is a Win7 x86 machine w/ a single CPU and 4gb of ram.

This image, the VM on is on the RAID...baseline, as you can see the performance is dismal.

This image, I turned on the cache that is on the hard drives (the hard drive cache, not the RAID card cache).

This image is the VM on a single SATA 160gb drive connected to the systems board's SATA controller.  Notice how a single drive is significantly faster.

This image is with a 512mb battery backed cache installed on the RAID card.

Desktop VM Optimization

Running Windows7,8, or 10 as a VM whether it be VDI or just a VM the OS is bloated and wasteful.  VMware has a fling that helps this, (VMware flings are items created by VMware but are not fully supported).  TheVMware OS Optimization tool analyses a VM and compares it to a list of best practices for optimizing resources.


VMware 6.0 U1 & NetApp upgrade random thoughts...

One of the downfalls of v6 is that the “Storage Views” tab is gone…..which I used primarily to identify stale VMware snapshots.  The alternative is to use PowerCLI:

OR I found this handy tool:

http://www.robware.net/  Among many other nice things, it does give a nice single pain to view all VM's that have snapshots.

Do you have an email domain with only two letters?  IE bob@AB.com?  If so configuring email alerts is an issue.  Setting the email address with a two letter domain name will error out via the WebClient.  It works fine via the C-Sharp client.

VMware VCenter v5.1 Update 1 will not work with Windows 2012r2; it must be update 3, VMware-VIMSetup-all-5.1.0-2308386.

The web client is light years better than v5.5 or 5.1.  IMHO it still needs work.  Logging on to it takes too long, it still drags, I find myself opening a couple of windows to the same vCenter to get the same info that I would normally get with the thick client.  The Web-Integration piece, is HUGE, the download is something like 100mb, I thought this was supposed to be a lightweight-client?

One of my machines had a problem with the Web-Integration services, it would install, but not register.  Even after installing it, the prompt to install it would show up on the logon page.  The solution was to download a slightly newer version of the client from vmware.com.

If one ever has SRM or any other tabs not showing up, restart the Vmware vCenter Web Service.
I have experienced where many options not showing up using a domain administrator account, but yet the options where all there using administrator@vsphere.local.   Chances are permissions got messed up.  Fortunately I had two sites to do a stare and compare between.

The NetApp Virtual Storage Console, is HORRIBLE!!

-There is no real c-sharp plug in, when one clicks on the plug in the C-sharpe client, it gives a message saying that the webclient must be used.
-SnapMirror jobs cannot be modified if a two character email domain is used.  The work around it to edit an XML file.
-The plug in often fails, and requires a bunch of tinkering to re-register it.  See https://vcenterfqdn/MOB
-SnapMirror backup jobs, there is only three statuses used, failed, successful, or completed with errors.  If one wants to dig deeper consult the email logs.

.....work in progress......

Re-sizing hard drives on VM's while using SRM

vSphere Replication does not support changing the length of a replicated disk.

Virtual machines that are protected with VMware's SRM v5.1 cannot have their hard drive re-sized when that VM is protected.  In order for one to change the drive size, on must:
1. rename the replica folder (if you don't do this the entire VM will need to be replicated again).
2. delete the replication job, for whatever reason on cannot simply just pause it.
3. use vmkfstools -X vmdkname.vmdk to grow the replica copy.
4. change the size of the hard drive on the VM.
5. re-setup the replication job.

Windows Hard Disk Cleaner install....

Windows has a fairly useful tool called to clean up wasted space on the hard drive.  Unfortunately on Server 2008, it is not installed by default.  One has to add in that feature; which isn't a big deal, but it also requires bit of "Desktop Experience" and "handwritting."  Now what hand writting has to do with cleaning temporary files off of a hard drive I do not know.  Either way, this annoys me.  Here is how to install it with out going through the a fore mentioned trouble.

Copy cleanmgr.exe from the winSXS folder to the Windows\system32 folder and cleanmgr.exe.mui from the winSXS folder to the Windows\System32\en-Us folder.   If the machine is 64bit, copy them from the AMD64 folder, and if it's 32bit copy it from the x86 folder.


So far the only issue that I have seen is that this doesn't create that button that says "Disk Cleanup" when looking at the hard drive.  The program must be manually launched from the System32 folder.


In Windows 2012r2; this procedure does not work.  It seems as though the files in WinSXS are compressed.  I ended up copying the two files from another server that did have the Desktop Experience installed.  However one only gets four categories to clean up; where as the one with the full desktop experience has over 7.

"Granger" over at ServerFaults.com posted this which is very handy:
If you want more "GUI-ness", proceed...
To get your new "Disk Cleanup" into the Start menu, create a shortcut to the "cleanmgr.exe" file. Put the new shortcut in the "%ProgramData%\Microsoft\Windows\Start Menu\Programs\Administrative Tools" folder; name it "Disk Cleanup". Now, when in the Start menu, you can just type "Disk Cleanup", and you'll have the option to start it as Administrator at the outset.
To get "Disk Cleanup" to show up as a button on a disk drive's Properties dialog box, you have to change the registry to add a new key and "Expandable String Value". Create a new .reg text file and paste in the following:
Windows Registry Editor Version 5.00



NetApp Log on issues.....

While trying to administer some Net App's I would get his unhelpful error message trying to log in.  I assumed it was a Java deal, and tired uninstalling/re-installing, various flavors, all w/o help.  The newest On Command System Manager 3.1.2RC2 is supposed to work with Java 8.x.   Oddly I could log onto the 3250's but not the 3250's.  All are running v8.2 7-Mode.  

Thanks to Cody's Blog for posting up the answer.  Turns out some Net Apps still use SSLv3, which is disabled by modern browsers, turning TLS on  (on both controllers) allows one to log on!
options tls.enable on

HP ProDesk600 G1

The HP ProDesk MINI is a full PC in the size of a thin client.  The one I am playing with came with an Intel Core i5 CPU, 8gb of PC3 (two sodimms, up-gradable to 16gb), a 120gb mSATA SSD drive.  It does have the option of adding a 2.5" hard drive.  I am impressed that even while working this machine hard, I never hear it, nor does the case get hot.

computer recycling

One of the things I do is recycle old computers for people.  There is some money in it, but by the time one factors in tearing them down, sorting them, storing them, and hauling them to a scrapper, one might have enough money to buy a beer.  I rather enjoy it, and I don't mind providing the service to friends, family, and clients.  I had a friend drop off a number of machines to me, and the following was in the pile!

I have reservations about scrapping it.  I haven't even seen one of these in over 10 years!  Call it nostalgia I suppose.