OWA logs user out when they try to set an Out Of Office Message

After upgrading to Exchange 2013 users can no longer set their Out Of Office (via OWA option 'Set Automatic Replies'). When a user tries to do so via OWA they are immediately logged out and get the error message "The user name or password you entered isn't correct. Try entering it again."

We are currently on CU3, not sure if the problem was introduced in Exchange 2013 initially or once CU3 was applied.

Users see the same behavior when they try to check a Message Delivery Report via OWA.

Stephen