c# – How can I make different task execute synchronously (waiting for previous to continue), when they are started from different threads?

I’m writing the server in C# for an online card game that is turn based. Actions in a game must be performed one at a time, and the next action must wait until the previous finishes.

These actions are triggered by different events (user action, timer finishing, user disconnected), which are fired asynchronously and can come from different threads.

Since this is the server, it will be running hundreds or thousands of games, and it’s imperative that each game can be executed asynchronously so that delays in one game don’t affect other games.
However, events for a single game must be run synchronously, in the order in which they arrived.

This is the GameExecutor class that I created to encapsulate how I want it to behave. There is one GameExecutor for each game.
AddAction will be called from different threads, and stores the actions in a ConcurrentQueue to be processed in order.
I use a semaphore to make sure ExecuteActions() can only execute one at a time (per game) even if multiple events are coming from different threads.

using System;
using System.Threading;
using System.Threading.Tasks;
using System.Collections.Concurrent;

public class GameExecutor {
    private ConcurrentQueue<Action> pendingActions = new ConcurrentQueue<Action>();
    private SemaphoreSlim semaphore = new SemaphoreSlim(1);

    public GameExecutor() {
    }

    public void AddAction(System.Action action) {
        pendingActions.Enqueue(action);
        ExecuteActions();
    }

    public async void ExecuteActions() {
        // Wait for the semaphore
        await semaphore.WaitAsync();

        _ = Task.Run(() => {
            try {
                // Execute actions
                while (pendingActions.Count > 0) {
                    Action first = null;
                    pendingActions.TryDequeue(out first);
                    first?.Invoke();
                }
            } finally {
                semaphore.Release();
            }
        });
    }
}

It is called like this

public void onGameMessageReceived(GameMessage gm) { // Comes from a websocket message
    Game game = getGameByUserId(gm.userId);
    if (game == null)
        throw new GameMessageException($"Game not found for user {gm.userId}");

    game.executor.AddAction(() => {
        GameMessageResult result = game.executeGameMessage(gm, true);

        if (result.isSuccess()) {
            runGame(game); // If user turn is over, this runs the AI turn.
        }
    });
}

This approach is not working for me, and with a lot of users it can get to a state where a single game is being executed on 2 threads at once, which results in various errors.
This contradicts what I understand of how SemaphoreSlim works, which I would expect to wait for the semaphore to be released after the previous action finishes.

Where am I going wrong? What approach should I take to achieve synchronism within a game, but asynchronism across different games?