In order to convey some background knowledge, we are a web design agency that places a high value on web design in our region. We had a lot of traffic from this and other smaller subdomains.
We host client test / review sites in a subdomain of our primary Web site domain (for example, client.example.com) to display their Web site to the client. After launching a client site, we made the mistake of not removing the DNS-A record pointing to a VPS we no longer own. Someone uploaded a lot of PDFs to this server, and all 40,000 files were indexed on Google as part of our domain (for example, client.example.com/file.pdf).
This happened around November, based on the Google Search Console. However, as we have been very busy in recent months, we did not notice it until our rank fell. First, it was during the Christmas holidays, when we were completely gone from Google for a few days, but then came back. And then again in early March, where we have no rank except for a URL for our own brand name.
I noticed this a few months ago and deleted the DNS record some time ago and made sure that the URLs returned an HTTP 404 status. Nevertheless, Google kept over 1000 indexed URLs. It was not until I submitted a manual removal request using the old GSC tool that they were removed. However, we know that they may come back for search, as this is not the purpose of the tool.
A few weeks later, and we're still not up to anything in the ranking like it used to be – despite healthy backlinks, domain age, etc.
Is this a Google penalty for the rogue subdomains, or could it actually be due to a change in the algorithm? We are completely lost, because the URLs are gone, our pages are actually indexed by Google – just no more ranking.
How do you recover best? Our thoughts are to rebrand on a new domain and move all our subdomains. We are also redesigning our website and rewriting the content. This could be helpful on the new website when algorithm changes are the cause. We believe that it is best not to use the Google address change tool to avoid penalties (if that was the problem).