I’m needing to break up my API calls into 2 different requests. Since the API I am using uses cursor pagination, I cannot simply pick a cursor to start from in the database and pull the second portion of the data.
The API allows you to put a filter on from a date in time but doesn’t allow you to use pagination with the filter still applied. Is it possible to only apply a parameter on the first query and remove it from the pagination queries afterwards? Or would it be possible to feed the “more” parameter for the next page of data into a seperate request?
Sounds like you likely want to use two separate “Pull from API”/ “Enrich with API” steps to complete this use case. From there, you could either use the ‘Stack Tables’ or ‘Combine Tables’ step to merge the results of the requests (if it makes sense to do so).
Does that seem like a viable option/ do you have any concerns about handling the requests in this fashion?
To answer your question, Parabola does not have the ability to apply different pagination settings to different requests within the same API step (unless you are passing the pagination info via the request body or as a parameter since you can use merge tags aka column referencing). If you did want to send requests with different parameters from one API step, you could create the API endpoints within a column of the dataset, and then reference those URLs as the ‘API Endpoint’:
For instance, you can create a column called “API” (likely with an If/Else step), where row 1 contains the endpoint URL https://api.com and row 2 contains https://api.com?test=true. Then, in your API Endpoint box, you could reference the column containing your endpoints by typing {API}. This will send a unique request for each endpoint in your table.
That being said, I do think using 2 separate API steps is the far better solution here if possible. Let me know what you think!
