I'll post a controversial response that many probably will not agree with. Is sitemap.xml still necessary? It is no longer decisive, no. Is it good to have it and does it help? For sure. Should you delete it? I can not say, because I do not know what your above reasons are.
View this article written by my former colleague with years of SEO, UX, and Web Dev experience. I agree with many things:
If you do not actively update your Sitemap and do not submit to GSC or Bing Webmasters, you probably do not need them. A sitemap is a suggestion for search engines how to crawl your site. So think of websites like news sites with hundreds of thousands of pages: Sitemaps have size limits. For news sites, they do not contain every published article. However, these articles are found, indexed and classified.
If your website is updated regularly; has a clear information architecture and category hierarchy; and contains a well-planned internal link structure. The search engines index and arrange the content independently of the existence of the sitemap.
Note, however, that you will lose the ability to manually submit the Sitemap to GSC, which will trigger a crawl again. You also lose the ability to suggest search engines which parts of your website are most important or which ones are current.
Many SEOs still say that Sitemaps are critical, but that's traditional SEO wisdom. What matters is not how I would define it; it is more helpful, as I have observed in recent years. In your case, it depends on your reasons for deleting and how much you currently manage.