Intercom API 2.0 - Pagination

I am connecting to intercom’s 2.0 API. I am running into an issue with pagination only returning one page of data. If I don’t use a the courser key “starting_after” than I will get duplicate results on each page, instead of getting unique results on each page. The way I am using the “start_after” key, I am only getting 1 page of results, even if I request to fetch more pages.

This is the request…
I wish I could view the request being sent when it doesn’t fail, but when I intentionally make it fail so I can get the request, this is what is shown (I don’t see any pagination parameters, but maybe that’s because it’s failing)

{
“data”: {
“query”: {
“field”: “custom_attributes.stripe_plan”,
“operator”: “~”,
“value”: “clinic”
}
},
“headers”: {
“Authorization”: “Bearer ABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890=”,
“Content-Type”: “application/json”
},
“method”: “POST”,
“params”: {},
“url”: "https://api.intercom.io/contacts/search
}

Any ideas on what I am doing wrong?


Looks like there is a discussion about this on intercom…https://community.intercom.com/t/api-2-0-contacts-pagination-contains-duplicate-values/1664/3

Hey Dan,

You need to change your Cursor Key to next.starting_after to access that key.

If that doesn’t work, then try setting your pagination key to starting_after in your API Import. Its unclear to me what the pagination key should be.

Hey Brian,

That makes sense…At this point, any Cursor Key that I use returns 1 page of results. This the very first (It’s an API Import) part of a series of chain of events. Something else that I notice is if I put page limit to 150, I am still getting 50 results…https://api.intercom.io/contacts/search?per_page=150

Not specifying a key returns duplicate pages…

Two other articles on intercom that provide ideas on what else to try…There are more than just these threads…
*https://community.intercom.com/t/list-conversations-page-size/825/2

Any other ideas on what I can try? If we can get this working, we’ll probably be between 6000 to 8000 credits per month.

Other things I’ve tried

Hey Dan, took a look at this for you. I was able to get it to work for the regular contacts API with these settings:

image

Feel free to replace the Max Pages to Fetch with whatever number you want. When you’re ready to view the whole data set, you can also add in a Top Level Key:
image

However, I’m having trouble getting it to work with the POST to /contacts/search. As best I can tell, Intercom seems to want the pagination data to come through in the body. However, we don’t support that today. Just in case you’re curious, that would look like this (but again we don’t support this today):

Are you able to pull in enough contacts via the all contacts endpoint to not need to do the search?

1 Like

Hey all,

Confirming what Alex found, which is that you need to send the pagination settings in the body of the request (which is odd). Right now, that is not possible in Parabola.

As suggested, you will need to pull in all contacts, not using the search endpoint, and then filter them down using a Row Filter in Parabola to mimic the behavior of the search endpoint.

Best,
Brian

We have over 700k contacts, so naturally I was trying to slice and dice per certain data criteria… with these limits we would burn through unnecessary credits when I only need to run 5k to 50k through a workflow, and it would take over an hour to process each request, which isn’t terrible, it’s more the economics I am dancing around.

Putting the per page in the body worked! So now I am getting 150 results, but I can’t seem to get the cursor working, I tried “starting_after”:"{pages.next.starting_after}", but it seems to be sending that verbetem instead of filling it with the variable. Like brian just interated, this is likely what isn’t possible at the moment:(

So I have to run it at 100 per page or I hit limitation errors. But now when I run to get all, it seems to be capping at 100k records

Is there any way to have this limitation of 1000 pages increased? I am trying to do get all 8541 pages 🥲: since I can’t do pagination in the body.

That is a lot of pages! Unfortunately I don’t have a way of bumping that page limit up. Is the 100 items per page limit the upper limit of their API?