plotting – RegionPlot and ColorFunction strangely inaccurate in Mathematica 12.3

Here is a simple example of a RegionPlot of a disk with a user-defined ColorFunction that is supposed to be rendered with high quality according to my expectation. However, both the disk boundary is not rendered very precisely (polygon shape is recognizable despite PlotPoints -> 100 and PerformanceGoal -> "Quality") and the color function is very imprecisely reflected as well. Is this a problem with Mathematica 12.3 or is the problem in my formulation?

pts = Thread(RandomPoint(Disk(), 10) -> RandomColor(10));
  ColorFunction -> Function({x, y}, Nearest(pts, {x, y})),
  ColorFunctionScaling -> False,
  PerformanceGoal -> "Quality",
  PlotPoints -> 100