Updating VMware vCenter and ESXi Certificates

By default, the installation of VMware’s vCenter and ESXi use self-signed certificates with hardcoded passwords to protect the private keys of their SSL web services. While it gets you services that work out of the box, it is really bad form and a poor security practice.

If you install (or update to) version 5.1 of the VMware infrastructure components, you will be left with a bunch of warning windows like the ones on the left. If you’re lucky enough to have access to your own public key infrastructure, you can issue your own certificates to replace those provided by VMware so you don’t see constant warnings. However, if you undertake this effort be forewarned: VMWare’s guidance (Replacing Default vCenter 5.1 and ESXi Certificates) is incomplete, inconsistent, and at times incorrect. This article describes the problems with VMware’s document, and the final section will hopefully help save you a little of the difficulty I suffered in updating these certificates.

Incomplete

The first thing that is problematic about VMware’s guidance is that it is incomplete. It leaves out an important detail: you have to use their hardcoded password. It says so right in this knowledgebase article.

Changing the keystorePass parameter from the default testpassword is not permitted. Also, the password used to generate the rui.pfx file must be testpassword.

While it is possible to change the passwords for many of the certificates for many of the services, you are wandering into unsupported territory. A grep search reveals at least 7 .xml and .properties files that need to get updated, and again you’re working against the advice of the knowledgebase article above. Only one time, for one certificate, does their whitepaper let you know that you must use their default password of testpassword.

Inconsistent

There are six services that are used as part of vCenter/ESXi v5.1, and each one needs its own certificate. The generation process for each is very similar, and really could be simplified if VMware were to create a script to automate the generation of certificate requests. Without a script to help automate the process of generating the certificate requests, the chances of an inconsistency causing errors being made increase greatly.

There are many other ways in which the document is inconsistent with itself:

• At times, the document assumes that all the services are running on separate machines. At other times, the document assumes that some of the services are running on the same machine.
• Sometimes, the document includes links to the prerequisite steps for each part of the process. Sometimes it does not.
• Sometimes, the document includes the paths to where the files exist on Windows 2003 server and on Windows 2008 server. Sometimes, it only includes one or another.

Incorrect

One of my favorite quotes from the document is here: After the initial restart of the services, wait for five minutes. If the VMware vSphere Profile Driven Storage service stops during this time, restart it. If you have done everything correctly except you have changed one of the default passwords without updating the appropriate .properties and/or .xml files, you can start this service, and it will stop within 30 seconds. So you read this instruction, and restart it again. And carry on with the rest of the document. Unfortunately, you will never get this service, or any of the other services which rely upon it, to start and run correctly.  The certificate password problem causes the service to fail to start, and while you can continue on through the rest of the document, the services will not work.

And if you’ve got a setup where most of your vCenter installation is on one machine (as I do), and you follow their instructions, you end up generating and overwriting most of your certificates, as the document expects them all to be in the same folder with the same name.

Final Suggestions

With that, here is my advice for a successful update to all the vCenter and ESXi certificates.

1. Password: As much as it pains me, stick with their default password of testpassword. In all cases, their instructions require you to also place the raw, unencrypted private key in the same location as the file protected with the hardcoded password. So, it is no worse than having the private key in its unencrypted form. Make sure you do set appropriate operating system controls on the files to limit who can have access to those files.
2. Organization: Create a folder called c:\certs (or something) and have 6 subfolders: InventoryService, LogBrowser, SSO, UpdateManager, vCenter, WebClient. Place all your .cfg files in these folders, and generate/place your .key, .csr, .crt, and .pfx files in these folders as well. This will help you stay consistent and make sure you don’t accidentally overwrite files.
3. Make backups: Make sure you never overwrite any of the files when you are putting new certificates in place. Always copy or rename the original files so that you can get things working again if one of the inconsistencies of the document causes you a problem.
4. Check Services: Anytime you’ve finished updating certificate files for one of the services, make sure the service will start, and run properly. If you start a service successfully, check on it again a minute later. If it has stopped, don’t continue blindly – check the log files for errors. Unfortunately the Windows Event Viewer will typically only tell you about service-specific error 1 (0x1) or 2 (0x2). You’ll need to check your logs in folders such as %ALLUSERSPROFILE%\Application Data\VMware\…\logs.

I hope these tidbits can help save someone else some time. If you would be interested in a script that could help automate much of this process, let us know in the comments!