Paginating the API Requests
Pagination is a common technique used in APIs to manage large datasets by splitting them into smaller, manageable chunks or pages. This user guide explains how to use pagination with the API query parameters provided, including limit, offset, order, and dir.
In the following examples the endpoint /api/resource/ is used to explain the usage of pagination parameters, but this endpoint is not available. Please see the API documentation for endpoints supporting these parameters e.g "Get the organisations for which the current user is a member".
Pagination parameters
1. limit
- Description: The limit parameter specifies the maximum number of entries to return in a single API response.
- Data Type: number
- Default: 1000
Example:
In this example, the API will return a maximum of 10 entries in the response.
2. offset
- Description: The offset parameter determines the starting point within the list of entries from which results will be returned.
- Data Type: number
- Default: 0
Example:
In this example, the API will skip the first 20 entries and return results starting from the 21st entry.
3. order
- Description: The order parameter allows you to specify the column by which you want to order the results. For example, you can order by the created or updated column. Check the API specification for options.
- Data Type: string
- Example Allowed Values: created, updated
- Default: created
Example:
In this example, the API will order the results based on the created column.
4. dir
- Description: The dir parameter lets you specify the sort direction for ordered results. You can choose either ascending (asc) or descending (desc) order.
- Data Type: string
- Allowed Values: asc, desc
- Default: asc
Example:
In this example, the API will order the results by the updated column in descending order.
Combining Pagination Parameters
You can combine these pagination parameters to fine-tune your API requests. Here's an example that demonstrates how to use all the parameters together:
In this example:
- limit=20
- specifies that you want a maximum of 20 entries per page.
- offset=40
- skips the first 40 entries, starting from the 41st entry.
- order=created
- dir=asc
- sorts the results in ascending order.
Getting all the results
To get all the data from a paginated API endpoint you need to loop until the number of returned items is less that the limit asked for. For example:
const maxResults = 1000
func GetIPsForScan(scanID string, apiKey string) (*IPAddresses, error) {
client := resty.New()
offset := 0
var ips IPAddresses
// get maxResults until we've got them all
for {
// get the results
resp, err := client.R().
SetQueryParams(map[string]string{
"limit": fmt.Sprintf("%d", maxResults), // Maximum number of entries to return
"offset": fmt.Sprintf("%d", offset), // Offset into the list of entries to return
"order": "created", // Column to order results by (<created | last_seen>)
"dir": "asc", // Sort direction (<asc | desc>)
}).
SetHeader("X-Hexiosec-API-Key", apiKey).
Get(fmt.Sprintf("https://asm.hexiosec.com/api/v1/scan_data/%s/ips", scanID))
// Check the err and response code
if err != nil {
return nil, fmt.Errorf("error calling API: %q", err.Error())
} else if resp.StatusCode() != httpStatusOkay {
return nil, fmt.Errorf("error calling API: got return code %d", resp.StatusCode())
}
if err != nil {
return nil, err
}
// unpack the response body, decode the JSON
var newResults IPAddresses
err = json.Unmarshal(resp.Body(), &newResults)
if err != nil {
return nil, fmt.Errorf("error unmarshalling JSON response: %q", err.Error())
}
// append them on
ips = append(ips, newResults...)
// check if we got them all
if len(newResults) < maxResults {
break
}
// increment the offset
offset += maxResults
}
return &ips, nil
}
type IPAddresses []struct {
ID string `json:"id"`
IP string `json:"ip"`
DNSSources []struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"dns_sources"`
DNSPtrs []struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"dns_ptrs"`
Services []struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"services"`
Certificates []struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"certificates"`
CloudRegions []struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"cloud_regions"`
Asns []struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"asns"`
Entities []struct {
ID string `json:"id"`
Name string `json:"name"`
} `json:"entities"`
RiskCounts struct {
Info int `json:"info"`
Low int `json:"low"`
Medium int `json:"medium"`
High int `json:"high"`
Critical int `json:"critical"`
} `json:"risk_counts"`
Country string `json:"country"`
City string `json:"city"`
Latitude float64 `json:"latitude"`
Longitude float64 `json:"longitude"`
Seed bool `json:"seed"`
Created time.Time `json:"created"`
LastSeen time.Time `json:"last_seen"`
}