When developing enterprise software based on Windows, the importance of storing encryption keys in Windows Certificate Store becomes an issue.
The vulnerability of not working with the Windows Certificate Store is storage of encryption keys on the file system and not in a secure location as defined by Microsoft. The risk is not that high since both NTFS file system and Windows Certificate Store can be protected by an ACL, however, this is a standard for the customers I work with.
- Each framework implements the SSL handshake distinctly, e.g. The implementation of the .NET framework is not the same as OpenSSL or Erlang.
- The solution should not be coupled with a specific protocol, e.g. HTML, AMQP or any proprietary protocol.
- The enterprise software should not be recompiled to overcome the challenge.
- Performance should be minimally affected (it's very hard).
- The solution should support both SSL and client-side authentication.
- Private keys must be stored in the Windows Certificate Store and marked as not exportable. In fact, preferably to store the private key on HSM, but it's just a matter of changing the Windows CSP (Crypto Service Provider).
Since the solution should support both access to the Windows Certificate Store and transparent protocols, the OpenSSL solution is not good enough due to inability to work with the Windows Certificate Store. In addition, reverse proxies with the ability to terminate SSL channels, e.g. NGINX and HAProxy, are also unable to comply with the requirements.
There is also an option to use WCF transportation, which works pretty fast, secure and it complies with certificate storage requirements etc. However, the main problem with this solution is that code changes required to replace the tunnel of the system, especially if the transport based on 3rd party solutions.
A walk-through the solution
The solution is pretty simple and even scalable. The main idea is to develop SSL termination socket proxy, which means that it is transparent for any application.
The architecture diagram illustrated below for Rabbit MQ, but it can work for any software:
Since Socket is lower than the application layer in the OSI model, it is much easier to control the flow in this level. Hence, the client should not be changed, except the port on the target server. On the server side, I would protect the Rabbit MQ server to refuse any connections from non-localhost address by configuration. As for the SSL termination socket proxy - it is responsible for all security requirements, e.g. Certificate revocation verification, mutual authentication, using the Windows Certificate Store to bind to the TCP listener etc.
As for the performance... If you develop in lower languages, it may be fast enough. On the other hand, if your'e and expert in non-blocking and fully asynchronous code writing, it can work fast enough even in .NET and Java.
I worked with two of my colleagues - Guy Baron and Nir Rotshtein - both are very experienced in software architecture and development. Folks - it's my pleasure working with you!