A few days ago I received a support request from a customer of our SFDC-PI-Adapter. The customer is a big company having hundreds of thousands of records on Salesforce. They implemented a query which would return a big number of records. At first they tried the Bulk API for the query. The problem was that Salesforce does not support foreign keys in Bulk queries, so you can use only one database table, which is not appropriate in the most cases. Despite an urgent need from developers, the support for foreign keys was not implemented by Salesforce yet.
The current solution is to use the REST API for queries. Salesforce allows use of foreign keys in REST API queries. The problem is however that it returns only 2000 records as response to the GET request. The next 2000 records are linked in the so-called nextRecordsUrl which is a part of the response. You can use it for the next request, which also contains a nextRecordsUrl and so on. The adapter uses the nextRecordsUrl to get all the data of the query result. The problem is that if you have a real big amount of data, the adapter have to make a big number of GET requests (one for each 2000 records chunk), thus the receiving of query result can take quite a long time.
One possibility to improve this is to use ORDER BY and LIMIT in your queries. By doing this you can order your result records, e.g. by Id, and use the biggest Id (if you use ascending order) as additional condition in your next query. This approach has the advantage that you will have well defined amount of records returned and nearly the same execution time for each query.
The current solution is to use the REST API for queries. Salesforce allows use of foreign keys in REST API queries. The problem is however that it returns only 2000 records as response to the GET request. The next 2000 records are linked in the so-called nextRecordsUrl which is a part of the response. You can use it for the next request, which also contains a nextRecordsUrl and so on. The adapter uses the nextRecordsUrl to get all the data of the query result. The problem is that if you have a real big amount of data, the adapter have to make a big number of GET requests (one for each 2000 records chunk), thus the receiving of query result can take quite a long time.
One possibility to improve this is to use ORDER BY and LIMIT in your queries. By doing this you can order your result records, e.g. by Id, and use the biggest Id (if you use ascending order) as additional condition in your next query. This approach has the advantage that you will have well defined amount of records returned and nearly the same execution time for each query.
Keine Kommentare:
Kommentar veröffentlichen