We recently discovered that our internal search engine (using elasticsearch) was picking up a last modified date from the header that did not match the actual date the node was last updated. It was way off. We use Cloudflare, and I speculate that is is putting out the last cache date instead.
How do we provide a true last modified date header?
Our current work around is to use a preprocess function to provide a meta tag that our internal crawler will read. We check the current node for the
changed field and set a changed date value from that. If it is not a node, it uses the current date. This work around is fine enough for our internal crawler because we can control what it’s looking for. However, that won’t help external search engines that are crawling our sites. So, I’m hoping someone might have an idea for a better Drupal way of handling this issue.