database – Double Publish is Required To Update Data When Node is Under High Traffic

We have a high-volume production site that is running into issues of serving stale data when a node is republished.

Description/Example Of Issue

I go into a node, create a new draft, then make some changes to the “title” and the “body”. If I click “publish” while at the same moment there is traffic on the node, the updates will not take effect. Often times, the URL alias will be updated to reflect the new title, but the actual page will not display the new title or the new body. Similarly, when viewing the node on the Admin Content page, it will still show the old title. If we go back in and publish the node a second time, the changes will then appear in the display as expected.

Replicating the Issue

We’ve been able to replicate this issue by using JMeter to simulate traffic on a page. In JMeter, we created a “Thread Group” with “Number of Threads (users)” set to “2” and “Ramp-up period (seconds)” set to “2”, then pointed it to a particular node, and clicked “start”. From here, we see the same result of changes not displaying until the node is published a second time.

Steps We’ve Tried

  1. We are on AWS and have a read/write database and a read-only database. We thought that there might be a delay in synching changes over to the read-only database. We edited the structure to test this theory, and this was not where the problem lies.
  2. We logged out the node data at various hooks within the entity lifecycle to ensure we had the correct data at each step of the way. In doing this, we found that the updated data is present and correct at every step.

Our Current Theories

The problem definitely seems to be caused by caching issues and/or some kind of race condition. In theory, whenever you save a Node, it should invalidate the cache for that node, so the display will reflect the updated data.

The issue seems to lie where the data actually gets written to the database and/or where the cache for the specific node is actually invalidated. This appears to be pretty deep inside of the core functionality for Drupal and we’ve had difficulty finding good documentation on where and how this happens.