I have opened a merge request to TiBs repository fixing the User-agent which should increase the number of request reconciliation services are allowed to do. That said when/if merged this would need to be configured and applied to each of reconciliation instances in use by their operators.
Even if deployed by the various operators, the errors probably wont go away given Wikimedia’s new limits that are being rolled out. Ultimately we should probably swap the bundled services with one hosted on Wikimedia Cloud Services.
Thanks for the patch. Martin opened https://phabricator.wikimedia.org/T419770 following the initial discussion on a few threads in the forum here. The situation is more complex both organizationally and network topology-wise than it appears on the surface.
It turns out that the old wikidata.reconci.link service was actually a load balancer and set of reconciliation service instance hosted on a private server, but that also could forward requests to https://wikidata-reconciliation.wmcloud.org/ hosted on the Wikimedia Cloud Services. This had been set up as a backup when the WMCS servers were being unreliable, but with the new aggressive rate limits, the WMCS exemption from rate limits makes it a much more attractive host.
Today the private server got reconfigured to redirect directly to https://wikidata-reconciliation.wmcloud.org/ which should work without rate limits, even without changes to the User-Agent, mitigating the pain in the short term.
The organizational issues are still under discussion. Depending on how they shake out, we may want to think about whether we really want to continue bundling a Wikidata reconciliation service registration. It might be better to have the users register the service themselves, so they know who's running it, what they can expect in terms of support, what the data privacy guarantees are, etc.