I suppose the least distorted rectangular part of a shot is defined by the focal length of the lens used, but I want to know which part is accurate or at least how that part is calculated.
I think you really have to start here:
What is the difference between perspective distortion and barrel or pincushion distortion?
… because there are different types of distortions and you really need to be clear before you can do anything about it.
The basic answer is that the perspective distortion depends entirely on the distance you have from the object. Therefore, the focal length plays only as large a role as it affects your image. For this reason, there is no clear line between "close-up of eccentric things that look weird" and something that our brain considers normal.
For distortions that are actually a lens flaw (pronounced "design trade-off"), there is no general answer since you need to characterize each lens (or find an existing "lens profile" that contains this characterization). For zoom lenses, this is likely to change as you change the focal length. However, this is due to the design of the zoom mechanism of each lens that is not native to the focal length,
Camera lenses that are sold for photography are usually intended for art rather than science. The design intent is to create an appealing rendering. Therefore, they are not always ideal when used as measuring instruments.
Back to the top: When you need to move the camera to fit more in the frame, it naturally means you change perspective. If your topic is not flat, there is nothing to do. If you cut out the parts that are distorted in relation to your first shot, you will cut out the information you got from moving. Your software can turn things around later to align them, but it can not compensate.
(The only solution is to take lots You can create images from slightly different views, create a 3D model, and then render the model instead of working with the lens's raw data. That's what our brains do.)