I’m developing a page analyzer – the site is VERY anti-scraping and appears to not show the newest data immediately when you load a page.
If you refresh the page, it doesn’t always update. It seems like it will only let you see the newest data every 10-20 minutes. Even a CTRL-F5 refresh doesn’t update it.
Opening a new incognito browser seems to get the newest data. But then you need to close the browser and re-open to get it again.
Using a VPN does not appear to affect it, so it’s not IP based. Logging in/out doesn’t change it so not account based – it must be cookie/cache based?
Any suggestions on ways to allow the page to continue to auto-refresh, without closing/reopening an incognito browser?
