Require wp-load.php from subdomain redirects to 404 page

I face a very complicated situation, which I will try my best to describe as accurately as possible.

  • First of all my website is behind Cloudflare.
  • I wrote an external .php script that loads wp-load.php, fetches some data
    from an API, processes the data, and then stores the processed data in WP to some custom post types.
  • As execution of the script usually takes longer than Cloudflare’s 100 seconds time-out, in order to evade the 524 time-out, I used what’s called a grey-clouded subdomain, as suggested
    everywhere on the internet.

The only difference was that I had to use an absolute path to my
wp-load.php file, like so:

 require_once '/home/cityp2020/public_html/wp-load.php';

Everything was working fine, until I used W3 Total Cache plugin to further enhance the website’s performance (it’s a very heavy and crowdy site). So, as soon as I setup W3 Total Cache to use memcached method, all hell broke loose regarding my custom script…

Now, whenever I try to execute the script by visiting its URL the site redirects to (the subdomain part is gone) and of course I get a 404 error… What’s more, I get the following error in my subdomain’s error log:

(04-Apr-2021 19:21:25 UTC) PHP Warning: Cannot modify header
information – headers already sent by (output started at
/home/cityp2020/ in
/home/cityp2020/public_html/wp-includes/pluggable.php on line 1299

where line 1299 reads:

$x_redirect_by = apply_filters( 'x_redirect_by', $x_redirect_by, $status, $location );
if ( is_string( $x_redirect_by ) ) {
    header( "X-Redirect-By: $x_redirect_by" );

header( "Location: $location", true, $status ); // <- This is line 1299 of course

return true;

So I wonder, in what way does memcached affect my site, making WP get confused, and what exactly fires that redirect from subdomain to mainsite?

After honestly a lot of research the closest solution I found was to cancel the redirect by adding add_filter( 'x_redirect_by', '__return_false' ); to my functions.php, but it didn’t change anything…

Any help would be greatly appreciated. This one’s puzzling me for a week already and I’m getting desperate…

If you need any further clarification, please let me know. TIA.