TL;DR: This BMC event is expected whenever the NVIDIA driver is not loaded (BIOS, boot, updates, driverless OS) and clears on its own once the OS and driver are running; no action is required.

Recommended action:

  1. Confirm the message matches this pattern. In the BMC event log (SEL or Redfish), the entry will show REDFISH_MESSAGE_ID: ResourceEvent.1.0.ResourceErrorsDetected with message arguments containing GPUGetCurrentSMUtilizationV2 and internal RM issues on <GPU index>.

  2. Check the timestamps. The entries should cluster around server reboots, BIOS sessions, or update windows, and stop once the OS finishes booting.

  3. From the running OS, confirm the GPU is healthy:

    nvidia-smi

    The GPU should be listed with a driver version, temperature, and power reading. If the GPU is assigned to a VM with Discrete Device Assignment (DDA), run this inside the VM instead.

  4. Confirm no other GPU sensor errors accompany it. Temperature, power, and inventory errors for the same GPU are not part of this pattern.

  5. If all of the above check out, no further action is needed. The entries can be acknowledged or cleared from the event log.

Why:

The server's BMC continuously polls the GPU out-of-band, even when no operating system is running. Most of that data (temperature, power, serial number) is answered directly by the GPU's own firmware and is always available. GPU compute utilization is different: it can only be reported after the NVIDIA driver has loaded in the operating system and initialized the GPU's resource manager. When the server is in BIOS, rebooting, or running without the driver, that one query has nothing to answer it, so the BMC records a failure. Once the driver is up, the query succeeds and the messages stop.

The event's resolution text ("Reset HMC, if the issue still persists, reach out to NVIDIA") is a generic string from NVIDIA's GPU management firmware component. It does not indicate a fault in this scenario.

Going forward:

Expect these entries at every reboot and during Azure Local update cycles (Solution Update, SBE, and CAU all reboot nodes through a driverless window). If your monitoring watches BMC or Redfish events, filter on this message ID and argument combination to avoid false alerts.

Contact DataON Support only if the message continues to appear while the OS is fully booted and nvidia-smi reports the GPU normally, or if it is accompanied by GPU temperature, power, or inventory errors from the BMC. Either condition indicates a different issue.

Optional details:

Static GPU telemetry travels over the PCIe slot's sideband management bus and is served by the GPU firmware without host involvement. Utilization metrics are produced by the NVIDIA Resource Manager, which on current-generation GPUs runs as firmware on the GPU itself but is loaded and started by the host NVIDIA driver. No host driver, no resource manager, no utilization data. This also means that with DDA, where the host has no driver bound to the GPU, the message can appear while the owning VM is powered off and will clear once the VM's driver loads.

Passive (server-edition) GPUs rely on the BMC's sideband temperature reads for fan control. Successful fan management while this message is being logged is itself confirmation that the sideband path is healthy and only the driver-dependent query is failing.