I live right near Penn Station. This is how I do it.
I take the the Long Island Railroad (LIRR) from Penn to Woodside. That’s a 15-minute journey, max, and the trains leave about every 20 minutes, max. If you buy a ticket before you get on the train, it’s about $8.
Then, I take the LaGuardia link from across the road at Woodside ($2.75, but you have to buy it in advance using the machine thingie at the bus stop). That’s about another 20 minutes. It runs 24 hours a day, 7 days a week.
If things are dire, I take an Uber or whatever car-service from Woodside. There are usually some waiting. The important thing is that the train gets you to the other side of the river and saves oodles of time. Timing it well, I’ve done this trip in 45 minutes, but I wouldn’t rely on that.
For Newark, I take New Jersey Transit to Newark Airport from Penn, and then Air Train.
For JFK, the LIRR to Jamaica and AirTrain.
I’ve stopped taking taxis to the airport at all, because mass transit is always, always, faster.