Allow application to react on the flask
Currently I'm developing a REST API where users submit tasks. These tasks are performed indefinitely, and it is literally impossible for me to guess a time when they are done.
When the users submit a task, I create a simple entry in a Redis cache that contains the start time, the job UUID, the data, and the current status. If a job is received by a group of queuing workers on a separate system, execution may take some time, or may take no time (1 second versus 1 day), as previously mentioned.
My current implementation works fine, but I'm trying to see if there's an alternative solution to tweaking things and reducing the number of calls I need to go from front to back.
The flow is thus essentially:
- The user hands over the order
- Cache entry created, user immediately confirmed with job UUID
- Workers start task
3A. Employees update the cache entry upon completion
- The user asks the endpoint if the job is complete (check cache, STATE var).
- Return user data.
5A. If not completed, ask again and again X times (after completing the previous request)
5B. When the process is complete, stop asking the endpoint when it is complete
This setup works fine, but it seems very wasteful to send several hundred requests every few minutes to check the status.
Any input would be appreciated. If this is not the right place to post, please let me know and I can remove / move it.