architecture – Designing a logic to receive ACK for every message sent in a non-blocking way

This is more of a design/architecture question: I have got a priority queue based timer implemented where each node in the queue has timer info i.e head, next node, expiry time, some timer message (<timerMsg>) etc, and the nodes arranged based on their priorities and expiry time.

So if there are 3 timers with the following expiry time and priorities

Nodes Expiry Times Priorities
T1 5 1
T2 5 10
T3 10 1

The priority queue would be initialized with nodes T2 -> T1 -> T3 forming a linked list.

It’s a periodic timer so if T2 expires, it’s deleted off the the list, and appended to the same list with an updated expiry time e.g now() + expiryTime, which is 10

Every time the node expires, the <timerMsg> is written to a message queue which is read by a different process, say, a consumer. And once the consumer receives it, it sends back an ACK to indicate the <timerMsg> was successfully received. An ACK needs to be sent by the consumer within X seconds, and no ACK was received, the same message needs to be re-transmitted over to the consumer.

Currently, I have the code working fine which only writes to the message queue but it’s got no ACK logic, which I need to implement.

Sorta pseudocode for the Producer code…

void callback(const char *msg)
   // writes <msg> to a mqueue

void *Producer(void *arg)
  PQueue *head = pQueue->head; 
  long now = getCurrentTime();
  long timeElapsed = now - start;
  char *timerMsg;

  if (timeElapsed >= head->expiryTime) // if node expired
     timerMsg = head->timerMsg;
     PopAndPush(head);     // delete the current head from the list, and add with an updated time


My vague idea:

The node that expires is pushed to a different list, say, “Waiting List” instead of getting appended to the same priority queue. And a posix timer is triggered to expire in X seconds, and once the timer expires, a callback is triggered which shall tell the Producer thread to either re-transmit the message or append the node back to the priority queue if ACK was received.

Question is, if the ACK is sent within X seconds, would the timer still wait for X seconds? Doesn’t sound an ideal approach.

Does having a state machine in Producer be a better approach?

Any help is appreciated