how do I make my code get the number of pages available in advance, and then start several tasks to get several pages concurrently? Example: example.com/api?id=1 ; example.com/api?id=2 ; … and get the response from the server, until the page returns an empty array: “”

Hey cipher welcome! Any good programming task starts by breaking down a problem into simpler parts. The first part is: How to fetch a single page? What have you tried so far?

I’ve already requested a single page, organized and separated all the information I want to handle, I only need now to request several pages

The request was to look at the code You have written… If You know how to fetch a page, You should be able to fetch given a page param. This should give You an offset and a limit. And count will give You the total page number. I am also doubtful about the use of Tasks to preload data when a cache …

https://github.com/hevertydourado/api-dienekes Here is the code. I’ll try to solve it by passing a parameter, but from what I’ve seen, it has about 10 thousand pages, would that be the best way?

It was not clear data comes from external API… I have implemented GenStage pipeline with HTTPoison to get this concurrency. But my latest web scrap was with Finch to download paintings/metadata concurently from WikiMedia.

There is this old post, I remember I liked it, but maybe now it’s a bit old… Ex Manga Downloadr, um exercício com Elixir Update 19/11/15: Neste artigo eu menciono algumas dúvidas que tive, então leia este e depois acompanhe na Parte 2 para ver como resolvi. Como exercício (e também por…

Elixir Programming Language Forum

Fetch data from multiple pages

Questions & Help Questions

benwilson512 July 30, 2021, 10:20pm 7

Ok, what have you tried so far?