Part of the problem is cabling. Until recently all 10gb cabling used an something called an SPF+ port. Those ports can use twin-x cables, or GBIC transceiver that converts to fiber optics or RJ45. The RJ45 transceivers can be had in both 1gbps or 10gbps flavors. Note if using 10gb over RJ45 one will need CAT6a, CAT6+, or CAT7 cables (the standards aren't really standards yet). Netgear has very recently released RJ45 10gb switches, they aren't cheap, but they are much more obtainable than buying Cisco.
Now the switching aspect. I have access to Some Cisco Enterprise 3750 48 port POE switches that have the 10gb SPF+ ports, four of them. However they are way overkill. They are noisy and consume about 90 watts of electricity at idle! I stumbled on a 4 port SPF+ switch with an RJ45 port from Microtek for less than $150! It has really good reviews. Which is likely the route (no pun intended) I will take.
Until I make that purchase I decided to experiment, can one cross connect two machine together using a twin-x cable? We used to do this way back when with Ethernet, it just required a cross over cable. Turns out one can! Two 10gb NICs and normal single twin-x cable, proper IP'ing and the two systems can talk. In my case, since this isn't easily route-able to the rest of the network, I have my two VMware servers doing VMotion over the 10gb, and all other traffic goes over the normal 1gb NICs.
VMware natively sees the NICs
SPF+ RJ45 Transciever
Twin-x cable
MikroTik CRS305-1G-4S+in Switch 1x Gigabit Ethernet Port and 4X SFP+ 10Gbps Ports, RouterOS or SwOS
A really good video I stumbled on while doing my research.
