Windows host compatibility settings for OpenVMS on x86 VirtualBox guests

Preliminaries

Hyper-V should be disabled by default on a fresh Windows installation. However, some system accessories also use virtualization services and individual resellers may have configured the Windows system such that these services remain active.

Host systems running Windows 10 or Windows 11 that are running or have previously run Microsoft Hyper-V hypervisor may fail the OpenVMS CPU feature checks. The issue is that certain CPU features are supported on the host system, but the guest system may fail due to lack of XSAVE instruction (the OpenVMS Boot Manager compatibility check fails).

This issue can persist even if the Hyper-V feature has been removed. This happens because certain Hyper-V virtualization services, which are also used by various other Windows accessories, interfere with VirtualBox. The VirtualBox developers are aware of this issue and are working to improve the interoperability with Hyper-V.

Disabling Interfering Windows Features

Hyper-V

1. Go to Control Panel\Programs\Programs and Features
2. Open the "Turn Windows features on or off" menu
3. Uncheck "Hyper-V" and "Windows Hypervisor Platform"

Note: on newer updates of Windows 10 & 11, simply removing Hyper-V via the features menu described above will not fully delete Hyper-V. For these cases, the solution is to uninstall it via an elevated (Run as Administrator) command prompt:

dism /online /disable-feature /featurename:Microsoft-hyper-v-all

Hypervisor Autostart

1. Run a Command Window, Windows Powershell, or Windows Terminal as administrator
2. At the Command prompt, execute the following command to disable Hyper-V:

   bcdedit /set hypervisorlaunchtype off

Windows Core Isolation feature (only applicable to Windows 11)

1. Search → "Windows Security"
2. Go to Windows Security → Device Security → Core Isolation Details
3. Set "Memory Integrity" to OFF
4. Shut down your Windows host system by executing the following command:

   shutdown -s -t 2

5. Power on and boot your Windows host system again.

   The XSAVE instruction should now be available to your VirtualBox guest.

How to determine if Hyper-V services impact your VirtualBox VM

When you launch a VirtualBox guest, look for the icon in the guest window status bar.

• A green turtle icon indicates that the VirtualBox host is running as a Hyper-V guest with diminished performance.
• An icon with a V symbol indicates that you are running directly on a VirtualBox host.
• View the log file VBOX.LOG:

1. In the VirtualBox Manager window, right-click on the virtual machine entry and select "Show Log" from the menu.
2. In the log file, search for “XSAVE”:
   • If it shows "1 (1)", your VM guest has XSAVE.
   • If it shows "0 (1)", your VM guest has Hyper-V services impacting it.
3. In the log file, search for “HM”. The following message also indicates that Hyper-V is active:

   {timestamp} HM: HMR3Init: Attempting fall back to NEM: VT-x is not available.

   {timestamp} NEM: WHvCapabilityCodeHypervisorPresent is TRUE, so this might work.