networking – How to connect to Ubuntu server from Windows 10 locally and securely?

So recently I brought a pc for pretty cheap to use as a Minecraft server. I plan to set it up with Ubuntu server. I want to be able to send commands to it (to start the server, change things, stop the server) and I want to be able to turn on the pc and turn it off. The thing is that I want to put this pc/server somewhere next to my router and I don’t want to have connect a moinitor and keyboard and mouse to it whenever I want to change something with it. I want to able to connect to it from my Windows 10 pc. Both the Windows 10 pc and the Ubuntu server will be at home (locally) but I want to be able to connect it without giving access to the people who connect to the game server. When researching this on Google I found out about ssh but every single article I find that tell you how to use and setup ssh seems to be using it for remote connection. I am also worried that by using ssh that once I give my IP address of the server to the people that will join it (via port forwarding) that they will be able to remote into the server and change things. Which is obviously not ideal and not that secure. So to sum up, I want to be able to connect to my server locally and send commands to it, etc. Without giving access to other people.
Many thanks in advance and I am sorry if I haven’t included enough information as I am new here and don’t really know to explain my problem.