11-18-2022 07:28 AM
I have a table which loads from a DB view. It takes a long time ( v. very quick for incremental, so hurrah for CDC! ) and just recently ( i.e. yesterday ) started erroring out when doing the full load. The error message includes this:
The data source is taking too long to process [FETCH_NEXT_RECORD_SET] request with guid
[fe9f21dc-8e51-476a-8701-700b7cecc00f] .
Please, increase max retries threshold property. Doubling the value should suffice.
You can add/update the property `data.agent.command.max.retries` in `service.properties` current value (10).
which ... OK. But before I go futzing around w/ settings I want to make sure I know what they are doing. Is this really just increasing max retries *with the same timeout* which seems like it would put more strain on the source without increasing my probability of getting rows back or is it increasing the timeout value for the initial query?
Further background - this view behavior is expected. When it was running full loads successfully it would start the same time as other tables, but wouldn't show any rows returned for a long time. So the timeout makes sense and the view is not mine so I can't control it from that side.
I did the search bar here support, and documentation but no luck going.
Any insight?
11-20-2022 05:01 PM
11-21-2022 07:30 AM
I looked in the docs here and searched for "timeout" and "extraction" "extraction" has some hits, but not a timeout setting that I can see.
11-21-2022 01:30 PM
Randy, the error message itself says it is the Data Agent that is timing our while it waits for a reply from the Data Source. Therefore there IS a max retries configuration that we can update, and support knows how to do this. That being said, there is some reason that the particular query is taking so long to return to the Data Agent after it is submitted to the data source. I would recommend having the customer assist in monitoring the incoming query request and then determine why it is running so long. I had a very similar issue recently on a query from ERP ACDOCA ledger table. It's a large table and bumping up the retries assisted in allowing the query to finish, but only because the Data Agent was able to wait longer for the results.
05-22-2023 11:18 AM
I am interested in this issue as well. I have a very large table that is experiencing this same error. @JeffW, please confirm that I need to open a ticket with support to make this alteration.