luns-attached-to-vmware-vsphere-6-0-may-remain-in-apd-timeout-state

Magnus Andersson mentioned in his blog a pretty important VMware bug including the corresponding KB article.

When an APD event occurs, LUNs connected to ESXi may remain inaccessible after paths to the LUNs recover. The 140-second APD timeout expires even though paths to storage have recovered. You see the following events in sequence in the /var/log/vmkernel.log:

  1. Device enters APD.
  2. Device exits APD.
  3. Heartbeat recovery and filesystem operations on the device fail due to "timeout" or "not found" or "busy".
  4. The APD timeout expires despite the fact that the device exited APD previously.

This is of course a real issue in production environments as the LUN doesn’t recover after being gone for some seconds, no matter if the LUN is back online within the 140-seconds timeout.

As the LUN is not accessible anymore all of your virtual machines are affected (down) and some hosts might not come back to be managed in vCenter. Of course the hosts will continue to operate but it happens quite often that they get disconnected in vCenter.

To resolve the issue, you need to kill the outstanding I/O to the LUN and reboot the ESXi Host!

Kill the outstanding I/O: http://kb.vmware.com/kb/2014155

LUN colllapse APD Timeout

Photo courtesy of sanbeiji(CC ShareALike)

RELATED ARTICLES

Save energy without reducing VM performance in your VMware vSphere cluster
16 August 2022
Over the last couple of decades energy consumption went up massively in every data center and while the…
Dennis
Metrics & Logs support for IoT - Bringing Secure Monitoring and Logging to the Edge
7 July 2022
Simple uptime monitoring for Internet-of-Things (IoT) is well-known and requires knowing if the devices are up and running.…
Dennis
Monitoring Azure SQL Managed Instance with Opvizor Metrics & Logs
17 January 2022
When you have critical applications and business processes that rely on Azure resources, it's critical to keep an…
Dennis

White Paper — Registration

You will receive the research paper by mail.

Codenotary — Webinar

White Paper — Registration

Please let us know where we can send the whitepaper on Codenotary Trusted Software Supply Chain. 

Become a partner

Start Your Trial

Please enter contact information to receive an email with the virtual appliance download instructions.

Start Free Trial

Please enter contact information to receive an email with the free trial details.

Subscribe to our newsletter