The API for my POS service is limited to 1000 results per call. I want to setup a daily routine that downloads all orders and related data and appends it to a google sheet that contains historical data. In order to accumulate the historical data, I will need to send an API call with the appropriate start time and end time, then repeat. It uses UNIX time, so determining the start and end times is a matter of a math. The question is, how do I iterate through each period?
While I have not used the webhooks feature, it occurs to me that the start and end dates could be integrated into the webhook URL and at the end of each flow, the url could be updated with the next period’s dates. It would then, theoretically, iterate through each period and append the data to a google sheet. Would this work?
Do you have a sense of what the ranges will be to avoid hitting the 1000 results per call limit? How many calls do you imagine having to make to accumulate all of your historical data?
If you’re able to share which POS service you use, we can check out their API docs as well.
Generally, I’d suggest the Enrich with an API step or the Pull from an API step for this.
I have decided to do break each year into 7 different files, one for each day of the week. That way I can build a flow with 52 API enhance steps where the date is incremented 7 days for each, append a google sheet, and combine them in a different flow as needed. If you can think of a better way to iterate through, that would be most helpful. I would love to see a for while or for if loop with some infinite loop error handling options.