Load Win10 on NVME/PCI-e on HP Z800

This will be part one of a multi-part series [fingers crossed] covering my attempt to modify a HP Z800 workstation into file, plex and web server housed in a XL-ATX full size tower with 14 drives. I have been dabbling in hardware and software mods for around 20 yrs and I know this is by no means difficult but as a DIY'er this is one of the more daunting yet doable mods at a low price and I believe the bang-for-buck is right up there for an hobbyist looking for something to do over the winter.

A little more background. I have a home lab server running a AMD FX8320 w/ 32GB ram, a M1015 crossflashed to LSI 9211-8i HBA IT mode hosting 13 drives for around 36TB or 24TB usable under Parity mode. This is all managed under Win10 Pro and Windows Storage Spaces. I've spent quite a bit of time reading about it over the past couple of years and despite its slow write speeds, it serves its purpose as a Plex Media Server and NextCloud host. After the recent Win10 1809 update, I started to notice the system starting to hiccup and I noticed a lot of errors within Event Viewer. Some I fixed thru tutorials and regedit, but others just won't go away...namely, IO and write errors to my array which is being updated with Seagate 8TB backup/SMR drives that are cheap TB-per-$ but slow as hell in write performance. Again this is not an issue for my media library mainly for Plex. So I started looking at upgrades.

I always wanted to try server grade equipment but was always put off by their price and footprint; you don't get much room to install racks in a condo. Anyway, I came upon these HP workstations that have server innards but in workstation form factor. The Z600 and Z800 offers dual Xeon's and room for lotsa ram and there's a batch on sale locally. Initially, I only needed a Z600 in its small form factor to play with but I kept reading about the slightly sexier Z800. Being blinded by the prospect of playing with new hardware, I didn't consider some important specifics, like UEFI vs legacy BIOS on these systems. But by this time, my mind was already going a mile a second and I've already plunged into this project.

First off, I was inspired by this post to attempt this.

Hacking the HP Z800 Xeon motherboard into a standard case
http://andybrown.me.uk/2intergrate014/11/01/z800/

Much smarter guys have done this awhile back and I'm just catching up. But not being an IT professional, I don't get access to this level of equipment until they come cheap on the used market. I ended up with a full system with its OEM case for around $600 CDN:

HP Z800
Dual Xeon X5680 @ 3.33 GHz [12 cores/24 threads total, 130w TDP each]
48GB DDR3 ECC 10600 RAM [upgradeable]

I got a barebones system and needed to add my own GPU and drives. I added a GTX 1060 3GB and a WD 256 NVME SSD on a PCI-e adapter. This is where it gets ...interesting. As the updated BIOS doesn't support NVME storage, I had to get creative with my kiddy-scripting, read: combining best practices and trial and error them into a workable solution. The following is after around 10 to 12 hrs of research:

Step 1.

After much searching, I finally chanced upon this site and this post that talks about using Clover, a Hackintosh tool to provide UEFI boot. I downloaded the tool, burned a USB and fumbled around with it and was able to read the transplanted WD NVME on the first try...sweet! But it was the Win10 install from the AMD rig and I want the system to be fully activated so I attempted to get Clover to read my USB Win10 UEFI install media. After many hours of trial and error, Iwas not and still am not able to get Clover to read my USB Win10 install.

[Guide] NVMe-boot without modding your UEFI/BIOS (Clover-EFI bootloader method)
https://www.win-raid.com/t2375f50-Guide-NVMe-boot-without-modding-your-UEFI-BIOS-Clover-EFI-bootloader-method.html

[Looking back now, I imagine that Clover is either missing drivers to read a GPT USB in UEFI or that the USB key isn't formatted correctly as GPT for Clover to see it. We re-visit this down below]

There are 2 posts in this thread that I need to acknowledge.

First, this post by Charlie confirming that it can be done:

https://www.win-raid.com/t2375f50-Guide-NVMe-boot-without-modding-your-UEFI-BIOS-Clover-EFI-bootloader-method-10.html#msg47359

Second, circling back, we have this post by Plawerth that actually summarizes how to structure the Clover USB:

https://www.win-raid.com/t2375f50-Guide-NVMe-boot-without-modding-your-UEFI-BIOS-Clover-EFI-bootloader-method-9.html#msg44263

I encourage anyone reading this to read and read Plawerth's post as using his file structure helped me narrow down where to copy the NvmExpressDxe-64.efi file. I originally got my transplanted NVME working by just copying NvmExpressDxe-64.efi from \EFI\CLOVER\drivers-Off\drivers64\drivers64UEFI\ to \EFI\CLOVER\drivers64\ and \EFI\CLOVER\drivers64UEFI.

Step 2.

Since a clean install won't work, I went back to a tried and true method of doing this a few yrs back...install Win7 first. This was painless once I dug out a Win7 install and flashed it onto a USB for legacy BIOS boot. I realized this from this post:

https://support.microsoft.com/en-ca/help/2990941/update-to-add-native-driver-support-in-nvm-express-in-windows-7-and-wi

Specifically:

"Method 2
Restart from media, and install Windows 7 from media to a disk that attaches other storage controllers (such as SATA).
Press Ctrl+Shift+F3 to enter audit mode during the Out-of-box experience (OOBE) process.
Install the hotfix package, and then restart the computer.
Reseal the system by using the sysprep - generalize - shutdown options.
Capture and move the generalized image to the disk that attaches the NVMe controller."

I realized that I might as well clone the WinX install over to the NVME, like what had happened originally with the transplanted NVME from the AMD rig. But Win7 doesn't support NVME natively.

Step 3.

After Win7 is installed on a SSD and activated, I proceeded with upgrading it to Win10 1809. A sidenote about 1809, it's buggy as hell, but my server updated automatically and the Storage Spaces got upgraded as well and it can no longer be recognized or read by earlier versions. You can easily get this media off of Microsoft's Media Creation Tool. However, since 1809 is so buggy, it's actiually been pulled by M$ as of this writing, so I had to figure out how to get it to download. I found this:

How to download Windows 10 version 1809 right now
https://www.ghacks.net/2018/09/26/how-to-download-windows-10-version-1809-right-now/

After the download, I flashed the ISO using Rufus onto a USB key and inplace installed within Win7.

Step 4.

After Win10 is installed, I can now see the NVME drive. Using a preferred disk cloner, I cloned the SSD over the NVME.

Step 5.

At this point, Clover will not boot into the NVME as it's a clone of a Win10 install in MBR. Again, I just wanted to plow over this with a clean install but I still can't. Good thing is that unlike years ago, you can now convert Win10 in-place from MBR to GPT for UEFI boot:

Shifting from BIOS to UEFI with the Windows 10 Creators Update MBR2GPT disk conversion tool
https://techcommunity.microsoft.com/t5/Windows-10/Shifting-from-BIOS-to-UEFI-with-the-Windows-10-Creators-Update/td-p/60064

I got the original tip from here:

https://audiocricket.com/2016/12/31/booting-samsung-sm961-on-asus-p6t-se-mainboard/#comment-3550482711

"takis Audio Cricket • a year ago
I managed to boot after cloning.
You need to:

1. clone windows with your favourite tool
2. convert your CLONED drive to UEFI using the new windows 10 command:
   mbr2gpt /convert /disk:2 /allowfullos
   ---PLEASE REPLACE DISK NUMBER WITH THE CLONED PARTITION!!!!!
3. boot as normal using your instructions."

However, this is not all of it.

Step 6.

Using a Clover USB modified according to the Clover/NVME thread, I was able to boot into the NVME thru UEFI. The above commands did the bulk of the conversion, but left me without a Recovery Partition on the NVME after the conversion. I got stressed a bit but found this:

Windows 10: Windows Recovery Environment
https://www.tenforums.com/installation-upgrade/108486-windows-recovery-environment.html

I ran these commands and was able to confirm in DiskPart that the ~500mb partition is now set as recovery:

reagentc /disable
reagentc /setreimage /path R:\Recovery\WindowsRE [this line might have errored out for me]
reagentc /enable
reagentc /info

This set of commands in an admin CMD window created the Recovery Partition. You can confirm this thru DiskPart:

diskpart
list disk
select disk #
list partition

Step 7.

I want Win10 to boot on the NVME like normal without me confirming the boot disk thru Clover everytime. I modified the config.plist at the following:

[key]DefaultVolume[/key]
[string]EFI[/string]

and

[key]Timeout[/key]
[integer]2[/integer]

This boots into Clover and gives me 2 seconds to choose or stop Win10 boot. However, I followed Plawerth's post and pared the config.plist file down to its bare essentials for Windows boot using the above substitutions and now it boots almost immediately into windows after POST.

Great success!

If you've read this far, I thank you. If I've missed something please let me know. The next step is to replicate the functions of the old server and migrating cards over. I'm already running into the next set of issues! And this is all before I test fit the mobo to the case I chose. Stay tuned!

tl;dr

1. Follow NVME/Clover thread to set up Clover USB.
2. Install Win7 on SSD thru legacy BIOS.
3. Upgrade Win7 to Win10, in-place within Win7.
4. Install cloner and clone Win10 SSD to NVME
5. Log into Win10 on SSD and convert NVME from MBR to GPT.
6. Use Clover USB to boot into Clover and choose, start Windows EFI to boot into NVME.
7. Use reagentc.exe commands to rebuild Recovery Partition on Win10 NVME.
   8 Clean up Clover boot USB files and leave the USB key in for every boot.

I hope this helps someone like myself.