What “Did we miss an interrupt?” means - nvme_timeout

"Did we miss an interrupt" mean?
https://lkml.org/lkml/2018/4/26/623
Question:
We are testing NVMe cards on ARM64 platform, the card uses legacy interrupts.
Intermittently we are hitting following case in nvme_timeout of drivers/nvme/host/pci.c
       /*
         * Did we miss an interrupt?
         */
        if (__nvme_poll(nvmeq, req->tag)) {
                dev_warn(dev->ctrl.device,
                         "I/O %d QID %d timeout, completion polled\n",
                         req->tag, nvmeq->qid);
                return BLK_EH_HANDLED;
        }

Can anyone tell when does nvme_timeout gets invoked ?
What does "Did we miss an interrupt mean" ? Does it mean host missing to service a interrupt raised by EP card ?

Answer:
I recall we did observe issues like this when legacy interrupts were used, so the driver does try to use MSI/MSIx if possible.
The nvme_timeout() is called from the block layer when the driver didn't provide a completion within the timeout (default is 30 seconds for IO, 60 seconds for admin).
This message you're seeing means the device did indeed post a completion queue entry for the timed out command, but the driver believes it was never notified via interrupt to
check the completion queue.
This means either one of two things happened: 
1，the interrupt was raised prior to the completion queue entry being written；
2，or the interrupt was never raised in the first place.
It might be possible to determine which if you can read the values from /proc/irq//spurious and see if the "last_unhandled" aligns with the expected completion time.

看来legacy interrupts（通过INTx）不稳定，而nvme驱动也正是先尝试优先注册MSI-X/MSI中断的。

Patch for Poll CQ on timeout in case of "we miss an interrupt":
commit 0b2a8a9f4b564c7d923597828d93cd1f69ce40e0
Author: Christoph Hellwig
Date:   Sun Dec 2 17:46:20 2018 +0100

    nvme-pci: consolidate code for polling non-dedicated queues
    
    We have three places that can poll for I/O completions on a normal
    interrupt-enabled queue.  All of them are in slow path code, so
    consolidate them to a single helper that uses spin_lock_irqsave and
    removes the fast path cqe_pending check.
    
commit 7776db1ccc123d5944a8c170c9c45f7e91d49643
Author: Keith Busch
Date:   Fri Feb 24 17:59:28 2017 -0500

    nvme/pci: Poll CQ on timeout
    
    If an IO timeout occurs, it's helpful to know if the controller did not
    post a completion or the driver missed an interrupt. While we never expect
    the latter, this patch will make it possible to tell the difference so
    we don't have to guess.

commit 3a7afd8ee42a68d4f24ab9c947a4ef82d4d52375
Author: Christoph Hellwig
Date:   Sun Dec 2 17:46:23 2018 +0100

    nvme-pci: remove the CQ lock for interrupt driven queues
    
    Now that we can't poll regular, interrupt driven I/O queues there
    is almost nothing that can race with an interrupt.  The only
    possible other contexts polling a CQ are the error handler and
    queue shutdown, and both are so far off in the slow path that
    we can simply use the big hammer of disabling interrupts.
    
    With that we can stop taking the cq_lock for normal queues.