Another interesting build. Dell has a line of GPU servers, and they are aged enough that they are on the 2nd hand market. The cost of PCIe Tesla's or equivalent are kind of ridiculous. However, Nvidia has another form factor, called SMX or NVlink. It has more bandwidth than PCIe 3.0. Because it has a completely different interface, it means there is less machines than can run this GPU, thus the prices are much lower. In this case I was tasked with converting one from PCIe GPU's to SMX GPU's. There is not a lot of information on these machines so maybe this post will help someone else.
Pictured with Nvidia SMX GPU's
PowerEdge c4130 vs c4140
Side by side picture of the 2400 watt power supply next to the 1100watt power supply. Note the 2400w power supply take a different power cord! It takes a C19 connection. As the power supply has a 16amp draw. The system will not run on two 1100 watt power supplies, well kind of. The system powers on, get through 1/2 of the posting process then powers down. Looking in iDRAC, no errors are logged! See this video:
System powers on, starts the posting process then shuts down.
Picture showing SMX2 system board with the GPU's.
To put this board in the bottom tray had to be swapped out. To swap the trays the front frame had to be removed. Which unfortunately meant there is now way to mount the power button, and LED indicator. I couldn't find a replacement component listed anywhere. I might be able to cut the old one down. For now the LCD board is held in with foam and a zip tie.
Official cable routing for PCIe and power cables, there is not circuit board connector between the SMX board and the system board.
Picture showing post power, and PCIe cables, and GPU's installed.
Notes:
-System MUST use a 2400 watt power supply, dual 1100 watt supplies is not sufficient
-the third PCIe slot, the riser card is the same as the PowerEdge r640
-the 2nd PCIe slot does NOT support PCIe fraction
-Does not support booting from NVMe; at least not from a PCIe->M.2 NVMe drive
-There is a "kit" to mount two 2.5" drives in the spot where the 2nd power supply would normally live, I just could not find the parts to purchase
-Used the same "modified" OCD SATA connector as other x40 PowerEdge servers
-tried both the generic and Dell cable and could not get it to recognize any SATA drives.
-There is a fan shroud that goes between the GPU's..couldn't find the part number and/or a place to purchase
-Doing Automatic Updates on Windows 2019 would cause it to bluescreen. My theory is that the Microsoft video drives conflicted with the Nvidia GPU's.
pcie cable 0y6tgj
pcie cable 0688n0
pcie cable 02f5p8
pcie cable 086khr
power cable 0ynp05
power cable 0528cn
power cable 0528cn
power cable 0ryj56
