How do I paginate through large record sets?
When you query the Validic API for new data, the return payload will include a "Summary" response at the beginning of the record, which includes the number of individual data records returned, noted in the "results" field attribute. By default, the number of records returned in each response is capped at 1000 records per page.
This means that for large data sets (perhaps due to a large user volume or a wide scope between the given "start_date" and "end_date" of your API call), you may have to paginate to the next set of records in the data payload to receive the full set. This is accomplished using the "next" attribute field in the Summary response. The value of "next" is the full HTTP resource that you can use to make an API request and retrieve the next set of records in the result set.
NOTE: You can control the number of records returned by adding a "limit" parameter to your API call. The maximum is 1000 records per page. Also note that the succeeding response may also contain a value for the "next" attribute.
Below is a sample response that would require pagination to access the next set of information.
{
"summary": {
"status":200,
"message":"Ok",
"results":155,
"start_date":"2014-08-01T00:00:00+00:00",
"end_date":"2014-08-06T23:59:59+00:00",
"offset":0,
"limit":100,
"previous":null,
"next":"https://api.validic.com/v1/organizations/51aca5a06dedda916400002b/routine.json?access_token=[FILTERED]&start_date=2014-08-01T00:00:00+00:00&page=2",
"params": {
"start_date":"2014-08-01T00:00:00+00:00",
"end_date":null,"offset":0,"limit":0
}
},
...
}