The good news is that this is probably not as big a deal as you think. There are a lot of articles (usually written by people selling VPN subscriptions) that make it sound super scary to leak you IP address. This article is a bit more neutral:
Exactly how much information can be learned from your IP address will depend on how your Internet Service Provider (ISP) has set up their network and how they buy and assign IP addresses to their users. The pictures in this article explain it nicely. Most home internet ISPs will assign a unique public IP to each home router, in which case your IP address is unique to you. But some ISPs will put multiple customers behind a NAT so that they all share one public IP address (in particular I think mobile carriers do this because the sheer number of mobile devices would quickly exhaust the 232 possible public IPs). So the first step is to check how your ISP assigns IP addresses.
Next, even if your IP address is unique to your house, that does not mean the attacker will be able to get your address from it. Attackers will look up your IP address in a GeoIP database such as this free one, which is a big lookup table of IP addresses to physical addresses. How does the GeoIP database know your address? It doesn’t. Usually they will know that, say Comcast has reserved a certain block of IPs for, say, customers in northern Atlanta, and so the GeoIP database will just have a pin somewhere in northern Atlanta. In my case, the GeoIP database I linked to has my location wrong by about 500 km.
TL;DR I wouldn’t worry too much about it.
I have already tried unplugging my router because I was told that my IP address would change.
Did it change? (You can see your IP address my typing “my ip” into google). If it did not change with a reboot, your router may have a very long lease time (sometimes weeks), but you could try logging into your router’s admin page and see if there’s a button to drop the ISP connection or reset your IP address.