VMware iSER iSCSI targets showing up as "unconsummed"

I was swapping out an ESXi host, and I couldn't get our iSCSI target to mount.  The paths and targets would show up, however it would say the volume was "un-consumed".   Which is odd as the datastores in question are in use by other servers and many VMs live on them.  I do have one storage server that has issues with it's identifier due to a signature mismatch, and I have to forcefully mount it.  However this time the same behavior wasn't present.

Normally from the ESXi shell I issue:

esxcfg-volume -l   this will spit out all of the volumes and some basic details visible to ESXi, normally I would see the volume in question, however this time no volumes where present.

Turns out the issue was MTU!  The switch ports, the ESXi kernel ports, the storage server were all set to MTU 9000.  However, I missed the virtual switch!  It was still at 1500, changed it to MTU 9000, rescan the storage adapter and all is good.

iSER & RDMA don't fragment their packets, they just drop them!  



Windows 2019 Software RAID

 Just a quick test...This server has U.2 Samsung MZWLL1T6HEHP NVMe drives, a fresh install of Windows 2019 server.  For the first test I took the three NVMe drives and created a dynamic volume (software RAID5).  Then I repeated the test of a single drive.  

three drive, RAID5 software

Single drive


SuperMicro servers featuring PCIe 5.0 and 200gbps NICs

 Hot off of the factory floor!  SuperMicro ASG-1115S-NE316R

AMD Epyc 9004 series CPU




e3.s form factor drives.  For the record I dislike this form factor because I fear the "fingers" the drive interface extends beyond the body of the drive.  I can see some of these getting broken by sloppy and mistreatment.



SuperMicro AI GPU Server

 

SuperMicro ARS-111GL-NHR  G1SM-G

Nvidia A02 72 Core GH GraceHopper 3.4ghz CPU

16896 GPU cores

https://www.supermicro.com/en/products/motherboard/g1smh-g

https://www.supermicro.com/en/products/system/GPU/1U/ARS-111GL-NHR



A little concerned about these human hair thick wires just ran across the front






The server is physically very long, some cheating to get the server rack door to close with 200gb DAC cables.

RAID5 vs RAID10

Quick and dirty test.  The Guinee pig is a Lenovo ThinkSystem SR655.  With an Avago RAID 930-24i card with 4gb cache.  Four Koxia 1tb SAS 12gbs SSDs.  VMware ESXi v8 is installed on to the host.  The four drives were set up into a RAID5 configuration, read ahead enabled, drive cache is enabled, cache policy is set to Write Back.   The RAID virtual drive was presented to VMware; a Microsoft Developer Windows 11 VM was imported to the datastore using thick provisioning.   Atto disk benchmark software was ran with both 4gb and 8gb tests.  Then, the VM, datastore, virtual drive, was torn down, rebuilt as RAID10, and retested.





I found the results somewhat surprising.   We often hear about the "RAID 5 write penalty", or "RAID 10 is just faster", etc., etc..  Well this test shows the opposite to be true.  The write speed on RAID5 is actually better!  One theory is that well, three drives are doing the work of writing in RAID5 where as in the RAID10 only two drives are writing.  


Burnt up Network Card?

 I have seen this in decades.  This Mellanox 100gb card was causing the server to be unstable.  Now, I don't know when the card went bad.  The server was repurposed, and the 100gb NIC was added.  So I don't know if my co-worker put the NIC in already burnt, or it burnt up in this Dell R620 Power Edge

Notice the discoloration in the lower right corner, and that capacitor is also a different color.




Enterprise SAS SSD vs Consumer SSD

 The time was allocated to do a quick test to compare consumer grade SATA SSD drives to Enterprise grad SAS drives.   The SAS drives are 12Gbps, the SAS card is also 12Gbps.  The tests were done on the same Windows 10 desktop with the same LSI SAS card, expect where noted a PowerEdge r730 was used.

Baseline...Samsung Evo 840 1tb SSD on 12gb SAS controller, Windows 10

Samsung MZL1ls960 960gb SAS on 12gb SAS controller, Windows 10

Samsung MZL1ls960 960gb SAS on 12gb SAS controller, on Dell r730 PowerEdge on Perc H730 RAID card w/ 1gb cache, test set to 2gb, Windows Server 2019


Dell MZ-1LT3T8c 3.8tb SAS on 12gb SAS controller, Windows 10


Impressive results!  12gb SAS SSD drives are very near NVMe performance, and nearly double that of the SATA drives.  Is the performance due to having more cache?  Or is it because the SAS drives are 12Gbps vs 6Gbps on the SATA drives?  It should be noted that during testing the SAS drives consume roughly 2 more watts at idle 5 more watts during the test.   The SAS drives were also MUCH warmer to touch where the SAS drives were ambient temperature. 

Dell x30 PowerEdge Servers running NVMe

 I always assumed that generation 13 PowerEdge Server (r430, 530, 630, 730, 930 etc.)  where not able to run NVMe natively.  Many people run the M.2 "gum-stick" form factor drives on PCIe->NVMe adapter cards. 

As it turns out one can!   The parts are not cheap, but it basically a special PCIe controller card (Dell P31H2 16 Port PCIe x16 Extender SSD NVME Controller), SAS cables, and a special back plane.  The PCIe card runs four cables out to "special-ish" ports on the back plane, and four of the 2.5" drive bays can now run U.2 NVMe drives.















Windows 11 & Hardware

 Windows 11 has several hardware requirements that if they are not met, one cannot install the operating system.   Or can you? 

  
Windows 11 requires the following minimum requirements:

-TPM 2.0 modules (Trusted Platform Module (encryption))

-1ghz 64bit CPU

-64gb hard drive

-4gb RAM

-Graphics card compatible w/ DirectX12 and WDDM 

-720p or better display

Seems more than reasonable, except, the real bugger for most of this is the TPM part.  That rules out basically any machine that is older an Intel I-series generation 8.   So let's say one has a computer with an Intel i7-7700? Let's say one has a Dell PowerEdge r640 server running VMware ESXI v7 with Intel Xeon 414r CPU's?  Well those machines are more than fast enough, but they doesn't pass the TPM check; so Microsoft says: "Go pound sand, and then go buy a new PC.  Also what if one is doing virtualization?    

Well on can bypass this hardware check. There is no downside to this; the machine will run Windows 11 without problems.  It isn't going to BlueScreen, it isn't going to be slow, it isn't going to blow up; well at least not because of pseudo hardware requirements.  I have done it probably a dozen of times and have yet to run into any issues related to the hardware requirements.  One machine has even been running for since early 2022.

After booting to the Win11 ISO, at the very first screen where it asks about Language, time/currency, and Keyboard.  Hit "shift" and "F10".  This will bring up a Command Prompt.  

In that Command Prompt window, type: "regedit"

In RegEdit,navigate to : HKEY_LOCAL_MACHINE\SYSTEM\Setup

Create a new key named: "LabConfig"

In LabConfig create the following keys of DWORD value set to "1"

-BypassCPUCheck

-BypassRAMCheck

-BypassSecureBootCheck

-BypassTPMCheck

Close the Regedit and Command Prompt windows and install Windows as normal.  

https://www.tomshardware.com/how-to/bypass-windows-11-tpm-requirement


Bonus:  If one is using a USB key to install Windows, make the changes, and those settings will stick on the install media, as it is "writeable".  So one only has to do these steps once.

Bonus #2: If one doesn't want to deal with associating the machine to a Online Microsoft Account, simply don't have the machine connected to the internet until one is done installing Windows.  In other words; don't connect it to the internet until after setup is complete.