Following the deployment of a number of updated .wsp files across a four-server SharePoint 2010 Farm (two Web Servers), I noticed some inconsistencies with the custom controls which leverage these solutions.
Each error, in the form of a correlation ID, was linked to the following exception within the ULS:
An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail. —> System.ServiceModel.FaultException: An error occurred when verifying security for the message
But, not everyone was witnessing these errors, infact it was only a subset of users. So why the inconsistency?
Using my local DNS hosts files, I was able to pinpoint which server was proving to be problematic as this allowed me to bypass the load-balanced address.
The error message is created when WCF is unable to verify the security of the message that was forwarded to the service, and this can be applicable to the server time skew not being in sync.
The server in question was indeed 8 minutes out (yes, that’s another story), but changing the time to its correct setting and reactivating the solutions did the trick.
So, even if your solutions appear to have been deployed correctly and Central Admin is not reporting anything to the contrary, check your server time skew if you experience inconsistencies.
SharePoint Uncovered 