We were able to identify and reliably reproduce the problem in out testing environment, it is related to the TLS-Offloading which interferes with the traffic in some cases and brakes the communication with sending RSET commands and not handling the responses correctly.
Thanks to all who contacted me directly and provided their logs and inputs, it was most helpful.
As a workaround we disabled StartTLS inbound again and are working on a fix so we can re-enable it as soon as possible.
Regards Marcel