The search service is not able to connect to the machine that hosts the administration component
I have discovered a problem with the Search Service Application in my DR farm. Any query in the Enterprise Search Centre produces an error:
Search has encountered a problem that prevents results from being returned. If the issue persists, please contact your administrator.
When I attempt to manage the service application within Central Administration I see the following message:
The search service is not able to connect to the machine that hosts the administration component. Verify that the administration component '7c8a54c7-0c9c-4afd-9442-b99c08852982' in search application 'Search Service Application' is in a good state and try again.
When I ran the Get-SPEnterpriseSearchServiceInstance cmdlet it showed the status as online. I also found Event ID 1365 logged in Application log so after a quick Google I followed the steps on this TechNet wiki article but to no avail. I had a dig around the logs (using ULS Viewer) and spotted this:
SearchServiceApplication::Execute--Exception: Microsoft.SharePoint.SPException: Tried IMS endpoints for operation Execute: Operation sent to IMS failed: Resource saturation, try again later.
I thought this may be due to under-spec’d hardware so I increased the RAM but this didn’t help.
After a few hours of looking through logs and trying different things I resorted to completely removing the Search Service Application (using this script) and creating it again (using AutoSPInstaller). I can only assume that some configuration became corrupt but I didn’t really have enough time to continue investigating.