GenServer advice needed - relationship between handle_call and handle_info that use the same state

[yurko]

Some background: I am using parts of a working app in another, bigger app so some solutions have to be refactored or reimplemented.

@Qqwy

Using ETS makes sense if you have a lot of data that you might want to search through.

I have lots of data, but it’s pretty uniform and I have to handle each entry all the time, no searching. Complexity of ETS wouldn’t be a problem though, because the current implementation already uses it, still makes sense to stick with GenServer’s state?

@Qqwy

Depends on what you want to archieve exactly: If you want the items to be processed and they can be processed completely separately, then you could use Task.async or Task.async_stream to do the job, which means that process_queue will not block for long.

I will use Task.async_stream here, there is no question about that The jobs are separate and each one might have to work with an external API so concurrent processing is a must, but still even asynchronously it will take some number of milliseconds to finish.

@Qqwy

On the other hand, if it was possible to process items completely separately from the start, one could create Tasks right away, and not need to bother to use a queue at all.

It was a “no queue” implementation originally I don’t want to bore you with too much details but here is one case that justified the use of queue: since I have to use an external API it might have temporal problems or even one time errors, so at times I need to fallback to a simpler handler for a specific task and at other times - just keep an item in queue and schedule it for the next run since the error must be temporary but I don’t want to block for long.

Anyway thanks for the feedback, it seems there are no obvious problems with the above solution and I might use it.

I might also stick with mutable ETS state, the only problem with it is that it’s somewhat verbose to iterate over. Which one would you go with, @Qqwy ?