Want to be able to change the SSL Certificates on your vCenter Servers and vSphere Hosts to properly signed CA certificates at the click of a button? Want to be able to automatically apply certificates to Auto Deployed Hosts? Need a solution that covers vSphere 4.0 through to 5.1? Can detect and alert you to expiring or revoked certificates and change them for you? Something that provides complete SSL Certificate lifecycle management for your vSphere environment with all the reporting, alerting and audit logs you’d expect? Then vCert Manager might be the tool for you and I’ve got a demo of the beta version to wet your appetite.

I wrote an article regarding a few months ago titled vCert Manager – Changing VMware SSL Certs Made Easy, which included a demo of a very early prototype that I presented at VMworld USA in August 2012. The prototype showed how in concept vCert Manager could automatically replace SSL Certificates in vSphere environments with integration back to a windows CA. A few months on now and VSS Labs, who is the company developing the solution, has released a demo of the beta version. The beta is now being tested by select customers who have signed up to the early adopter program. Check out this demo to see how the product has progressed and get an idea of it’s capabilities.

I think you’ll agree that the vCert Manager Beta has come a long way since the very early prototype I presented at VMworld and wrote about previously. The work is not over though. There is still a little way to go to get everything feature complete and enterprise ready and before it is ready to be generally available. The work is progressing well. The demo below is only 16 minutes long, so won’t take up much of your time.

Final Word

I know how much pain changing SSL certificates is and the goal of vCert Manager is to take this pain away and provide a solution that makes it affordable for all vSphere environments to become more secure. I’d greatly appreciate your feedback on the vCert Manager beta demo. Your suggestions and feedback will help VSS Labs make vCert Manager a great product.

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2013 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.

I was upgrading my VMware View environment recently from 5.0 to 5.1 and wrote about some initial problems in my article Trouble Recomposing View 5.x Desktops After Upgrade to vSphere 5.0 U2. After I had resolved those initial problems I needed to load my internal Root CA certificate onto all my company’s iPhone’s and iPad’s. This is because one of the big changes or improvements in View 5.1 is with security and you now need trusted certificates in order to connect to any of the desktops. Fortunately there is no need to purchase expensive public certificates if you have an internal corporate PKI / CA’s already configured, unless you want to. This article will show you how you can easily get your iPhones or iPad’s to trust your corporate CA certificates for use with VMware View.

I’ve included images here to explain the process as I think it’s easier to follow. I used one of my iPhones to keep the images reasonably small. To be honest you’re much more likely to be doing this on an iPad. But iPhones are perfectly usable in my opinion provided  you have the iPhone to VGA adapters and a Bluetooth Keyboard.

Trying to Connect Without Trusting the Certificate

If you try to connect to a VMware View 5.1 environment using the iOS View Client without first trusting the CA certificate you will receive a message as per the image below:

If you click on View Certificate you will see some details about the untrusted certificate:

There is no way to set your device to trust your CA certificate from this screen. In order for you to get your iPhone or iPad to trust the certificate you will need to follow the process below.

Getting Your iPhone or iPad to Trust Your CA Certificate

1. Obtain a copy of the CA Certs (Root CA and Intermediate CA if used) and email them to your device, such as in the following image:

You’ll notice the attachment in the image above shows a certificate type icon.

2. You now need to tap on the attachment. You will be presented with the following screen:

At this point before continuing  to the next step you should click on More Details. You should verify that it is indeed the certificate that you were expecting, it’s form your corporate CA, and that it is valid and should be trusted. Once you are satisfied this is indeed a legitimate certificate that you should trust you continue.

3. Tap Install.  You will see the following warning image displayed on the screen:

Because your corporate CA is not a trusted public CA it is not automatically in the trusted list for your devices. This is the reason this warning is being displayed. Provided you are happy with the checks you’ve done in the previous step, after reading this warning you can continue to the next step.

4. Tap Install. You will see the following image displayed on screen:

At this point you need to enter your passcode so that the certificate can be loaded into your devices trust store and be trusted. Once you have entered your passcode successfully you will automatically be at the next step.

5. You have successfully loaded your corporate CA certificate into your devices trust store. You will see the following image displayed on the screen:

Now when you connect using the VMware View Client your Connection Servers certificates, which were signed by your corporate CA, will be trusted and your connections will be successful. If you have  more than one CA that needs to be trusted you need to complete these steps for each of the certificates. You can now Tap Done and go back to the VMware View Client and test the connections.

6. Now when connecting to your VMware View Connection Servers or Security Servers an image similar to the following will be displayed on screen:

You can see by the tick on the padlock and the text https being displayed in green that the certificate and connection are trusted. If the connections weren’t trusted you wouldn’t have been able to connect. Enter your username and password and then tap done or go.

7. You will receive the list of entitled desktops similar to the image below and you can no proceed to use your desktops as per normal. This process is complete!

Removing a Certificate From Your iPhone or iPad Trust Store 

If for some reason you find out that a certificate has become invalid or has been revoked you will need to remove it from the trust store on your iDevice. To do this is very simple.

1. Tap Settings.

2. Tap General. You will see on the screen something similar to the following:

You can see the profile listed and the name of the CA in this example.

3. Tap Profile. You will see on the screen something similar to the following:

4. Tap  Remove. You will see a warning displayed similar to the following:

5. Tap Remove. You will see the passcode dialog box displayed as per the image below.

6. Enter your passcode. You will be returned to the settings screen and you’ll notice as per the image below that the profile has now gone.

You have now completely removed the certificate from your devices trust store. When the new certificates are issued you can go back and follow the process to install them again.

Final Word

As you would expect Apple has made it fairly painless to get this all working. However when it comes to security and trusting certificates great care needs to be taken. You must verify that the certificates that are being sent to you for use are genuine and can be trusted. If for some reason the certificates expire, are revoked or for some other reason invalidated then you need to follow the process to remove the certificates from the trust store and then install the new ones. I hope this has been helpful and that you get hours of productivity out of your VMware View 5.1 vDesktops from your favourite iDevices.

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2013 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.

Microsoft recently issued a security advisory and a patch that blocks any certificates with a key length less than 1024 bits. This has impacted a wide variety of systems including VMware Site Recovery Manager 5.0 and below. This article will provide you a way to quickly regenerate the self-signed SRM certificates.

Background

The Microsoft Advisory and related information can be found in Microsoft Support KB 2661254. I would encourage you to read this article as the impact is wider than just VMware SRM and other VMware products.

Although I’m going to show you a way of generating self-signed certificates here for the use with SRM I would recommend using trusted CA certificates if possible to reduce the risk of man in the middle attacks. However the effort required to set up a CA and issue the certificates is far more than what I’m about to explain. This is the quick way to work around this problem until you come up with a better solution, which may include getting CA issued certificates.

Note: This process is not officially supported by VMware and says as much in the output of the command. So use this at your own risk and I would encourage you to test it in an isolated environment prior to applying this to any production system. Always take a backup of existing certificates before making any modifications. These instructions should work for any versions of SRM 5.1 and prior.

Generate New Self Signed SSL Certificates for SRM

There is a file called CertGenUtil.exe that is shipped with SRM and used by the installer to create the default self-signed SSL certificates. The version included in 5.0 and prior only generates 512 bit keys, which are not sufficiently strong after you’ve applied the MS patch. The MS patch requires 1024 bit keys or higher. Fortunately the version of CertGenUtil.exe shipped with SRM 5.1 generates 2048 bit keys and can be used to re-generate the certificates for use with versions of SRM include 5.0 and prior. You may need to use this if you are upgrading from SRM 5.0 to 5.1 also as the certificates are not generally replaced during an upgrade process. I have not yet tested the upgrade process of SRM 5.0 to 5.1 to see if it’s any different to previous versions with regards to the update of the certificates.

To use CertGenUtil.exe you will need to create a short config XML file so that it will generate the SSL Certs Correctly. The following is an example:

<config>

<DR_CERT_SERVER>SRMSERVERIP</DR_CERT_SERVER>

<DR_CERT_ORG>YOURCO</DR_CERT_ORG>

<DR_CERT_ORG_UNIT>YOURORG</DR_CERT_ORG_UNIT>

</config>

Replace SRMSERVERIP with the IP Address of your SRM Server, YOURCO with your company and YOURORG with your OU. Save the config file in an easily accessible location on the server where you’ll install SRM 5.1, such as c:\ or c:\temp, in the example below I’ve saved the file as srm-certcfg.xml in c:\temp. Note: you only need to install SRM 5.1 to get the CertGenUtil.exe, you are not required to upgrade your environment to SRM 5.1. So it would pay to do this in a test environment with a SQL Express instance and a test VC.

By default the CertGentUtil.exe file is located in c:\Program Files\VMware\VMware vCenter Site Recovery Manager\bin

On a server installed with SRM 5.1 or that contains the SRM 5.1 CertGenUtil.exe file execute the following command:

c:\Program Files\VMware\VMware vCenter Site Recovery Manager\bin\CertGenUtil.exe -cfg c:\temp\srm-certcfg.xml

You will notice this line appears immediately:

VMware internal use only. This program is intended for use only by the SRM installer.

Follow the on screen messages that are displayed.

Installing The New Self Signed SSL Certificates

After the certs are generated you need to install them in the trusted certs store of both SRM Server and also both of the vCenter Servers (Protected and Recovery Sites). This is as simple as logging into the systems as administrator coping the new Cert file across and double clicking it to install it in the cert store (Follow the wizard). You will need to go through SRM and do a ‘Modify’ install and use your new certs in .p12 format. You may need to restart the SRM Services on both SRM Servers before the new certificates will be loaded into memory.

Final Word

I hope this helps if you quickly need to regenerate the default self-signed SRM Certificates for 4.x and 5.x to be compliant with the new MS patch. I would recommend that you use CA signed certificates to improve security and reduce the risk of man in the middle attacks, so this should be viewed as a temporary measure. This should allow you to continue to run your existing systems till you are able to upgrade to SRM 5.1.

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2012 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.

Recently I wrote about Updating CA SSL Certificates in vSphere 5.1 which applied to the Windows installable version of the vCenter 5.1 and it’s supporting components including SSO. VMware has now also released the instructions to update the CA SSL certificates in the vSphere 5.1 vCenter Virtual Appliance.

While there are a total of 136 steps in the process to update the CA SSL Certificates in the Windows vCenter 5.1 there are only 81 steps to update it in the vSphere 5.1 vCenter Virtual Appliance. But note that these steps do not include update manager. I will include the link below to the KB regarding update manager also.

Here are the KB articles required to update the vSphere 5.1 vCenter Virtual Appliance and Update Manager.

Configuring certificates signed by a Certificate Authority (CA) for vCenter Server Appliance 5.1 – http://kb.vmware.com/kb/2036744

Configuring CA signed SSL certificates for vSphere Update Manager in vCenter 5.1 – http://kb.vmware.com/kb/2037581

It was another great team effort across the globe within VMware to put these instructions together and test them. Hopefully you find this information useful.

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2012 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.

Over the past few weeks I have been working behind the scenes with a team of people at VMware spread around the globe on the process to successfully change out the self-signed certificates in vSphere 5.1. With the introduction of Single Sign-On in vSphere 5.1 the process is somewhat more complicated than vSphere 5 (ok quite a lot more complicated). But now I’m able to bring you some of the solutions you’ve all been waiting for.

This work covers vCenter, and all the related core components such as SSO, Inventory Service, Update Manager etc. The great news is that this work has resulted in KB’s that I and a number of others have tested and verified to work with vSphere 5.1 GA for the Windows installable version of vCenter. There are also updates to some previously released KB’s for vSphere 5.0. These processes will also work with the recently released patches to vCenter. The KB articles for the vCenter Virtual Appliance edition will also be published shortly and I will update this article when they are available.

Below are the links to all of the articles and a note with regard to Update Manager. I want to say a massive thank you to all of the people at VMware that made this happen. It was a big team effort. I’m glad I could make a contribution to the effort.  I will be making sure the process is automated for you as part of the vCert Manager project that I’m working on. My goal would be to automate both the Windows Installable and Virtual Appliance editions for vSphere 5.1.

Note you should start with KB 2034833 – Implementing CA signed SSL certificates with vSphere 5.1.

VMware has also put out a blog article on these KB’s titled Implementing CA Signed SSL Certificates with vSphere 5.1.

Note: I have found a problem with Update Manager when vCenter system is an all in one configuration with everything on the same VM and using a local MS SQL Server database. Update Manager will not be able to log into or register with vCenter when the SSL certificates have been changed. This prevents you from updating the SSL certs for Update Manager and Update Manager may no longer work. This does not appear to occur when the MS SQL Server database is remote. I have not tested this with a local Oracle or other supported local database. I am continuing to work with VMware on this issue and will update this article when it is resolved. In the meantime I would recommend placing the databases for vCenter and it’s other core components on a separate VM, even in small environments.

Final Word

Although changing out the self-signed SSL Certificates is not simple, and is very time consuming to do manually, the above articles make it possible and give you a tested and verified process. I will be automating the processes to take this pain away as part of the vCert Manager project. In the meantime I would recommend you start with KB 2034833 – Implementing CA signed SSL certificates with vSphere 5.1 and work your way through the rest. I hope you get a lot of value out of these articles and the effort that the team has put in. As always your feedback is appreciated.

Derek Seaman has put together a great series of articles on VMware vCenter 5.1 Installation that includes coverage of SSL certificates. I would highly recommend you check it out. Derek has made a great contribution to the process for SSL Certificate Replacement in vSphere 5.1.

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2012 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.

The twitter wires and blogosphere were ablaze with news out of VMworld US 2012 (August 27 – 30th). This was my first ever VMworld (with hopefully many more to come), and I greatly enjoyed it and I also enjoyed meeting many of you. My direct flight home to Auckland from San Francisco on Air New Zealand was the best flight I’ve ever had, and I got a full 8 hours sleep so I didn’t have any jetlag (Thanks Air New Zealand). But this article is all about my take on the event, what I learned, and vSphere 5.1. I’ve decided to do something slightly different to others, to take it all in, and then write this roundup post VMworld. I’m also going to target this towards the relevance to production and business critical applications environments. I’ll also give you some insight into the sessions I presented, the results and my lessons learned.

vRAM and Cloud Suites: Ding Dong! The Witch is dead. Which old Witch? The vRAM Witch! (The movie this is from is well before my time – Can you guess which movie?) Yes the vRAM Witch is now definitively dead. VMware Announced in the first Keynote on Monday 27th August that vRAM licensing is no more and they will instead be introducing vCloud Suites containing a bundle of products based on per CPU Socket, unlimited cores, unlimited RAM, and unlimited VM entitlement per licensed socket. Although the suite is a collection of products right now over time will become ever more integrated, and the licenses can’t be broken apart into their individual components. This is all very good news for VMware customers. The vRAM announcement was made in the context of vSphere 5.1, but it also applies to vSphere 5. So even if you’re running a vSphere 5 environment vRAM is no longer relevant. Personally I didn’t have a problem with the concept of vRAM as everything is moving towards a consumption based model, but it did cause a lot of extra things to consider during design, especially when every customer I ever engaged with had no impact as a result of vRAM. The free VMware vSphere Hypervisor will still be limited to running on hosts with 32GB physical RAM, but there are no longer any vRAM limitations (think configured memory / overcommitment is unlimited). Everyone with vSphere Enterprise Plus will get entitlement to vCloud Suite Standard, and VMware will be running promotions to get customers to upgrade to the other editions, so watch out for those. Information and comparisons between the vCloud Suites. This is great news for all environments, especially those with Monster VM’s. So now you don’t have to worry about VMware vRAM licensing for your business critical applications you can go back to only worrying about your ISV licensing and the best solution to meet all your other requirements.

vSphere 5.1 and the Mega Monster VM: 64 vCPU, 1TB RAM, 1M IOPs per VM, less network jitter, lower latency, Zero-downtime upgrade for VMware Tools (from 5.1 onwards), Dump Collector works with vDS. These are just some of the highlights of vSphere 5.1. VMware has taken the Monster VM and turned it into a more Mega Monster VM. Best of all the efficiency is still what you come to expect form VMware. So if you can configure 64 vCPU’s you know you can get within a few percentage points of native. IMHO it’s not good saying your architecture supports a huge number of vCPU’s if you can’t utilize them efficiently. VMware does a lot of work to ensure they optimize their architecture to get best efficiency as well as scalability. 1 Million IOPS per VM is great if you have a single VM that you can run off an entire fully FLASH array. But realistically this is just to eliminate any possible thoughts that the hypervisor is the bottleneck when it comes to storage. The test was conducted with 4k IO size and produced very low latency. The IO Size and Latency being important factors as I outlined in Storage Sizing Considerations when Virtualizing Business Critical Applications.  See What’s New in VMware vSphere 5.1 – Platform and What’s New in VMware vSphere 5.1 – Performance.

VMware vSphere Distributed Switch: Config Backup/Restore, Rollback and Recovery, Network Healthcheck, BPDU Filter. There is now no reason to run a mixed vSS / vDS environment. With the config backup / restore and automatic rollback and recovery you can be confident that the vSphere Distributed Switch (note name change) will be reliable and available, and easy to recover when things go wrong. The Rollback and Recovery will revert any change that has the consequence of disconnecting the hosts from vCenter or vice versa. Network Healthcheck will periodically check the network for configuration errors such as incorrect VLAN trunking, incorrect MTU, uplink erros etc and alert you to these issues before they become a major problem. This should greatly reduce the effort required in quality assurance when provisioning new hosts and operating hosts as the environment changes. The new vDS also supports Netflow v10 (IPFIX), LACP (IP Hash Only), and also RSPAN/ERSPAN. BPDU Filter is important as it stops the accidental or malicious configuration of a bridged VM from causing a physical host port down event and cascading failure across the cluster. BPDU filter will filter out any BPDU packets. As mentioned above the Network Dump Collector, which collects Purple Screen of Death (PSOD) Core dumps from ESXi hosts now works with vDS. In vSphere 5 this only worked with the standard vSwitch. See What’s New in VMware vSphere 5.1 – Networking.

VMware vSphere Storage Enhancements: All Paths Down (APD), Permanent Device Loss (PDL), Storage IO Control Enhancements, Parallel Storage vMotion, Combined vMotion / Storage vMotion without Shared Disks. The APD and PDL behaviour has been again enhanced in vSphere 5.1, which will see far more predictable behaviour under what should be very rare storage failures. Storage IO Control has been enhanced to be more self tuning. Storage vMotion now supports up to 4 parallel disk copies per VM. See What’s New in VMware vSphere 5.1 – Storage. With the combined vMotion / Storage vMotion and no need to have shared storage we can say goodbye to the concept of a swing datastore or jump datastore. Duncan Epping does a great job of covering this in his article “Say Goodbye to the Transfer LUN aka Swing LUN aka Stepping Stone“.

vCloud Networking and Security: HA, SSL VPN, Load Balancer, 10 NIC’s per Edge, VXLAN Gateway, Endpoint included with vSphere 5.1. All of the new features of vCloud Networking and Security are a major leap forward from the previous version of vShield, which this supersedes. The HA functionality for vShield Edge combined with support for 10 NIC’s, which are user configurable between internal / external means that you can realistically replace a large number of enterprise firewalls very cost effectively. This also means you can very cheaply set up realistic testing and validation environments to test multi-tier applications and their firewall rules before you apply the firewall rules to production physical firewalls. With HA if one host with the primary Edge device fails the firewall state will failover to stand by Edge, this is a real active / passive firewall cluster. Load balancing has been greatly improved to include health checks and can now support HTTPS pass through and any custom TCP ports. SSL VPN is a very convenient way of allowing end user access to the vApps and infrastructure protected by the Edge or for management of the infrastructure. The admin user interface has been greatly enhanced and so has it’s capability, including the logging functionality. Many will be pleased that rules now have a rule ID and this flows through into syslogs. The interface is much more intuitive when it comes to App Firewall also and is simplified removing the rule precedence that existed in the previous version. Flow monitoring is improved and you can now get statistics per rule to determine which rules are being used in addition to the top rules that are used. With Endpoint now included with the Hypervisor I predict that most organizations will start moving to VMware’s Endpoint protection and partner integrated solutions. Service Insertion now allows parters to integrate virtual editions and physical editions of their components with vCloud Networking and Security and also vCloud Director. This will allow many organizations to further differenciate their services and offerings.  The automation capabilities that are possible through vCloud Director, vCloud Connector and the REST API’s mean that vCloud Networking and Security is a major step forward with capabilities that really deliver on the software defined datacenter and software defined networking and security. See VMware vCloud Networking and Security Overview.

vCloud Director: SDRS Integration, Storage Profiles / Storage Tiering, Elastic VDC, Linked clones on VMFS across 32 hosts, vApp Snapshots, HA Edge Devices. See What’s New in VMware vCloud Director 5.1. There are so many improvements in vCloud Director 5.1 that I’m only going to cover a few very briefly. Storage DRS and Storage Profile integration is a big one. You will no longer require a separate Provider VDC just to support a different tier of storage. For smaller environments this made the design very tricky as you might in a single 2 or 3 node cluster have to support 2 tiers of storage. This forced you to break with some best practices and use resource pools instead of clusters as the demarcation for the Provider VDC compute resources. This will help greatly improve resource utilisation efficiency in vCloud Director environments. It will be interesting to see the new designs incorporating this and how they are now differentiating their service offerings. With vCloud Director 1.5 you could configure an Elastic VDC across multiple clusters only with the PAYG resource model, but all the vShield Edge devices stayed in the original cluster. With 5.1 you can now do this also with the Allocation Pool resource model and vShield Edge and system resource pools will be split across clusters. With the addition of VXLAN it is now also easier to stretch VDC’s across clusters and this adds improved performance to the isolation networks.

New Certifications: VMware launched a number of new certifications for the Desktop and Cloud tracks. We now see certification paths right up to VCDX-Cloud and VCDX-Desktop. The existing VCP and VCDX have been renamed slightly to VCP-DV and VCDX-DV to designate Datacenter Virtualization. The actual path to VCDX-Cloud and VCDX-Desktop is not quite clear yet and neither is the migration path for existing VCDX qualified individuals. But it is great to see these two new certification paths that will allow everyone to demonstrate their mastery of these technology areas in addition to Datacenter Virtualization. See VMware Certification Roadmap. If you think this looks similar to how Cisco’s certification works you’re right. This is intentional and it just happens the man who designed Cisco’s certification tracks is now in charge of doing that at VMware.

Oracle Virtualization Architecture and Performance Deep Dive: I presented two sessions at VMworld US regarding Oracle Virtualization. The first one APP-BCA1432 – Virtualizing Oracle Across the World — Success Stories from University of Auckland and Indiana University covered the process of how to go about virtualizing Oracle when migrating from traditional Unix platforms and how to engage the DBA’s and keep them happy. My content was based on a large project that I had delivered on behalf of VMware Professional Services. I had Don Sullivan (Oracle Certified Master) from VMware and Dan Young from Indiana University as co-presenters. In my second session APP-BCA1624 Virtualizing Oracle: An Architectural and Performance Deep Dive we really drilled into how to architect Oracle databases for maximum performance and how the hypervisor helped. In this session I had Mark Achtemichuk from VMware (Performance Technical Marketing) and Don Sullivan again. I took the same project as my previous session but this time really drilled down in the technical details of how we delivered 5x performance improvement from the source systems and as such a high ROI. Both would give you a very good understanding of how you really can virtualize Oracle Databases in large organizations successfully and ensure you meet the business requirements and performance requirements.

I received some pretty good ratings (4.39 and 4.3 respectively) for these sessions so a big thank you to all of the people that attended these sessions. You all thought we hit the mark with the content. This is very encouraging and I’ll try and do even better next year if I get a session selected. Based on the feedback a lot of people thought the sessions weren’t long enough. We could have talked for a lot longer and gone a lot deeper. This is the challenge when the sessions are only 60 minutes.

Automating Security and Compliance with DR: I presented this session INF-SEC1282 Automating Security and Compliance with Disaster Recovery Using VCM, vCOps, vShield, VIN and SRM along side Gargi Keeling who is the Product Manager for Security at VMware. This presentation was loosely based on a customer project I had been involved with where we had designed automated security and compliance processes along with DR. In addition to the learning from the actual customer project we enhanced the presentation with a partner solution (Catbird) that allows for automated syncing of vShield polices across multiple datacenters. The presentation covers all of the process and technology steps you need to take and gave an example of a technical architecture that would allow you to implement this, all using out of the box functionality from vShield, vCenter Configuration Manager, vCenter Operations, Virtual Infrastructure Navigator, and vCenter Site Recovery Manager, and supplemented if required with the Catbird solution. This presentation was also the worldwide premier of the SSL Management Solution mentioned below vCert Manager, which was very well received.

I received pretty good rating for this session of 4.15. Not quite as good as my Oracle sessions, so I will try and do better next time. This one was pitched as just a technical session not advanced technical. I also received a lot of feedback that the session wasn’t long enough and it would have been good to have the time to go deeper. What I’ve learned from the presentations I gave is that I probably need to narrow the scope and go a lot deeper. This will allow a lot more to get into a 60 minute presentation. Feel free to comment on this article and let me know your thoughts on this.

SSL Management – vCert Manager: My demo of the vCert Manager prototype was very well received and everyone in the audience of the Automating Security and Compliance with DR session agreed it would greatly simplify the process of managing SSL Certificates in VMware environments. I have published the Demo online and written about it in article vCert Manager – Changing VMware SSL Certs Made Easy.

Top Sessions I Attended:

The below sessions I highly recommend you review. I attended these sessions and thought they were a real highlight. Note I only had very limited time so I wasn’t able to attend many great sessions. I would have liked to have gone to the vCenter Technical Deep Dive and also Jason Nash’s vSphere Distributed Switch Deep Dive also. Jason got the top session of VMworld this year. I think it might be the first year a non-VMware employee has had the top spot.

Virtualizing SQL 2012: APP-BCA1516 Virtualizing SQL 2012 : Doing It Right. Jeff Szastak of VMware and Michael Corey of Ntirety managed to get through 160 slides of a very entertaining and deep technical presentation in just 60 minutes. I think they finished on time to the minute even with questions. I was very flattered that Jeff and Michael borrowed a quote from my Oracle Virtualization Architecture and Performance Deep Dive – “Your database is just an extension of your storage”. It is definitely relevant to SQL just as it is to Oracle or any other database. Optimizing storage performance is critically important and Jeff and Michael covered it well in the context of SQL Server 2012 and the relevant best practices.

SMP FT a.k.a. Multi-vCPU Fault Tolerance: INF-BCO2655 VMware vSphere Fault Tolerance for Multiprocessor Virtual Machines—Technical Preview and Best Practices. Presented by Jim Chow, Shrinand Javadekar, Srinivas Kotamraju, all from VMware. There was no timeframe or commitment given on when or if this might actually make it into the product given given how good it was I really hope it’s sooner rather than later. One of the attendees said this technology would literally save peoples lives as he worked in the 911 system as a systems admin and they could not leverage VMware FT currently due to it’s limitations. I can see many and varied applications for this. I can’t wait to get it into my lab environment when if it gets released.

Stretched Metro Clusters: INF-BCO1159 Architecting and Operating a VMware vSphere Metro Storage Cluster. Duncan Epping and Lee Dilworth did a great job of covering all the key points of architecting and operating a vSphere Metro Cluster environment. This is becoming a very popular solution for many environments these days, but it is not without its challenges.

Storage DRS Datastore Clusters: INF-STO1545 Architecting Storage DRS Datastore Clusters. Frank Denneman and Valentin Hamburger highlighted a number of key considerations when architecting Storage DRS datastore clusters, including some important limitations and considerations around storage IO control and array auto tiering. I wouldn’t operate a Storage DRS Datastore Cluster environment without reviewing this session first.

Oracle RAC Cluster Build Automation: APP-BCA1333 Virtualizing Oracle RAC. Rick Lindberg, Don Sullivan and Bryan Wood of VMware took the audience through the ins and outs of successfully virtualizing Oracle RAC on vSphere. Including the fully automated deployment of a new Oracle RAC Cluster in under 30 minutes (cut down demo recording was 7 minutes). The automation, which is available via a VMware Professional Services engagement allows not only new Oracle RAC Cluster creation but also node addition and node removal from existing clusters that have been created through this process. This will be especially valuable in Test and Development environments. The session also covered what VMware IT is doing in the process of virtualizing all their Oracle RAC systems and the necessary best practices to ensure the process is successful.

Final Word

It was great to see Oracle actually had an official presence at VMworld this year. They had a booth in the Solutions Exchange, which I stopped by for a chat and they gave me a nice T-shirt, and also had taxis and branded cars taking customers from their hotels to VMworld. This is another great show of support for VMware, which is a great place to run Oracle databases and applications. Oracle also confirmed at VMworld that running their applications and databases in a large cluster and using DRS Must Affinity Rules is a perfectly acceptable solution, provided the rules are not violated and the Oracle software is not installed and/or run on an unlicensed host. They also completely clarified the support situation with VMware vSphere. I think all of this is absolutely great news for Oracle and VMware customers. Now if you don’t believe that this actually happend why not just review the video, which is in an article on the License Consulting blog – VMworld TV – Richard Garsthagen Oracle Licensing and Support in VMware Virtualized Environments.

This was my first ever VMworld and it will definitely not be my last. I had a great time presenting to over 650 people and got great feedback. I met so many great people and was able to hang out with some of the VMware virtualization royalty. The only problem I had with VMworld was that it went way too fast. Mind you it was really hard work getting up at 6am every day and not getting to bed until after midnight most days. I would like to once again thank everyone that attended my sessions and gave feedback through the surveys, it was greatly appreciated. I’m looking forward to seeing some of the great people again in a couple of weeks at VMworld Barcelona, which I will be presenting a session titled APP-BCA1751 – Oracle Virtualization: Caging the Licensing Dragon with a great lineup of co-presenters. I hope to see some of you there. I also hope to meet a lot more new people.

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2012 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.

The vSphere 5 Security Guide has been officially released. There are a number of changes and enhancements and you should go through each to review the applicability to your environment and compare it to the vSphere 4.1 Hardening Guide. Since the public draft there have also been some significant changes that you should take time to review.

Before we go into the guide and my take on some of the new aspects I want to make you aware of a recent announcement that vSphere 5 has achieved Common Criteria EAL4+ Certification. This is an important benchmark and milestone for vSphere 5. This gives assurance that vSphere 5 can be configured in a secure manner and the security functionality is effective as per the software design. This is particularly important for hypervisors that will run multiple Guest OS instances on them, as they must ensure the isolation and security is enforced as designed.

The hardening guide is now delivered in a spreadsheet which is much easier to use and include in other documents. It’s very easy to follow, sort and search.

vSphere 5 Hardening Guide Official Release

VMware Official Announcement – vSphere 5 Hardening Guide Released

I previously wrote about some of the important changes in the new hardening guide vs the vSphere 4.1 hardening guide in my article vSphere 5 Security Hardening Guide – Public Draft. I would encourage you to review that article if you haven’t already as the points are still very valid. Since that post there have been some additional changes made to the hardening guide to include my recommendations around SSL certificates in particular, as well as clarification around some of the options that impact functionality.

I would like to draw your attention to the vCenter SSL Certificate recommendations in particular. Additional recommendations are made to check the validity of certificates and also to remove any expired or revoked certificates from your environment. These are very important administrative tasks that should be done if you are using custom SSL certs in place of the default self-signed certs. In my previous post I have linked to William Lam’s blogthat contains scripts to help you automate this task. If you want a way to fully manage the certificate lifecycle and replace certs automatically then you’ll want to check out vCert Manager – Changing VMware SSL Certs Made Easy.

William has also updated his vSphere 5 Hardening Guide Script, which will check the options against the guide and also check your certificates. You can find William’s script at the following location:

virtuallyGhetto: vSphere Security Hardening Report Script for vSphere 5

One of the reasons this is so important is that it protects you from possible man in the middle (MiTM) attacks. Another reason is because vCenter and the vSphere Client does not programatically check the validity of a certificate that it already trusts. It is once trusted, always trusted, unless you remove the trust. Without these important administrative tasks vCenter and the vSphere Client will continue to allow access without warning to any component with a previously trusted yet expired or revoked certificate. However any component that leverages Internet Explorer (such as performance overview and many of the vCenter plug-ins) will start to display warnings or cease to function if the certificates expire or are revoked, this is due to the checks that Internet Explorer does on the SSL certificates.

By considering security in your architecture design, making your designs secure by default and taking into account the appropriate level of hardening from the vSphere 5 Hardening guide you will have the best possible chance of limiting any security risks in your environment. Every environment has security risks, it is up to you as the administrator or architect to ensure you have the appropriate configuration, tools, controls and processes in place to limit the risks and balance security with functionality.

Now that we have the official vSphere 5 Hardening Guide I’m sure we will shortly see the vSphere 5 Hardening Template for vCenter Configuration Manager (vCM). If you don’t already have vCM as part of vCenter Operations Manager Suite – Enterprise I would strongly encourage you to purchase vCenter Operations Enterprise (include Operations, Capacity Planning, Virtual Infrastructure Navigator, vCenter Chargeback and vCM), or purchase vCM separately, or at least try it out in a proof of concept implementation. It will allow you to automate your security hardening and reporting across your vSphere Environment, as well as giving you visibility of configuration drift. vCM isn’t just limited to vSphere environments though as it supports native OS and physical systems (Traditional Unix, Linux and Windows). It can provide a one stop hardening, change management, compliance and audit/reporting shop for your environment, or at least the important parts of it. As well as physical bare metal OS provisioning and OS patching.

I hope you get a lot out of the hardening guide a lot of people at VMware have spent probably thousands of man hours compiling it and testing the recommendations and many of us interested parties have provided feedback to try and make this guide as good as it can be. Let me know what you think about the new hardening guide, I’m always keen to get your comments.

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2012 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.

The vSphere 5 Security Guide has been released publicly in draft form for comment. There are a number of changes and enhancements and you should go through each to review the applicability to your environment. Here is one of the highlights of the new version from my perspective and links through to the documents. It’s hard work putting this hardening guide together so thanks to Charu, Ben, Grant and Kyle, and the rest of the VMware Team for all their hard work on this.

I think you’ll really like the new format. It’s now delivered in a spreadsheet which is much easier to use and include in other documents. I would encourage you to review the drafts and to provide feedback directly to VMware via the community threads linked below, where you can also download the hardening guide.

vSphere Hardening Guide: 4.1 and 5.0 comparison – Rev B

vSphere 5.0 Hardening Guide – Public Draft

Duncan Epping has also published an article on the release of the public draft available on Yellow-Bricks – vSphere 5.0 Hardening Guide public draft available.

William Lam has just updated his vSphere Security Hardening Script and it is available in his article vSphere Security Hardening Report Script for vSphere 5.

One of the most important changes in my opinion is the removal of the recommendation to disable the VIX API from each VM in the VM configuration. This change has been replaced by controls being recommended in vCenter Server that prevent unauthorized administrators from making use of the API, while still allowing it’s functionality where necessary. This is a good change that balances functionality with security, and I’m very pleased to see it.

You may remember that I recently commented about the VIX API impact on SRM in my article – vSphere Security Hardening Policy and SRM 5, and this was also picked up on by Tech Target in VMware SRM 5 encounters potential security conundrum. There is now no longer a conflict or conundrum between the hardening guide and the requirements for SRM to re-IP VM’s during recovery. The use of the VIX API can be restricted to the SRM Service Account only, so that only this account, and not any human interactions (except for the supreme Administrator) can call it.  It can further be restricted to only the VM’s that require SRM to change their IP’s during recovery, by choosing where to apply the permissions. This makes it very easy to audit.

I was fortunate to be able to provide input into parts of the hardening guide while in ‘beta’ effectively, and I will be providing further feedback on the public draft. From my perspective I think it should include more recommendations regarding SSL certificates. I think SSL Certs, given the importance and difficulty, needs a bit more of a mention, especially around expiry and validity checking.

William Lam at virtuallyGhetto has written a couple of very useful blogs on the topic of SSL Certificates that you may like to review. I hope that the recommendation to check expiry makes it into the final version of the hardening guide. If you want a way to fully manage the certificate lifecycle and replace certs automatically then you’ll want to check out vCert Manager – Changing VMware SSL Certs Made Easy.

Extracting SSL Thumbprint from ESXi – virtuallyGhetto

Automating SSL Certificate Expiry Validation for vCenter Server + ESX(i) Hosts – virtuallyGhetto

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2012 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.

Changing SSL Certificates in vSphere environments can be difficult and error prone. But when I come across a problem where a customer is not able to successfully update their certs it’s generally down to one of four things. This article will provide you with the top 5 mistakes that are commonly made and how to avoid them.

If you want a way to fully manage the certificate lifecycle and replace certs automatically then you’ll want to check out vCert Manager – Changing VMware SSL Certs Made Easy. This will completely automate the SSL certificate process in vSphere environments. This will remove the chances of making most of the top 5 mistakes listed below. 

Top 5 Common Mistakes when Implementing CA Signed SSL Certs in vSphere:

1. Not following the correct procedure or steps, or missing out a step
2. Missing attributes from a Certificate Signing Request
3. CA Certificate Templates not configured correctly
4. Not using text transfer mode when copying cert files to ESX/Linux systems
5. Not having the full CA key chain in the certificate

Not following the correct procedure or steps, or missing out a step

For this you could be forgiven. The vSphere documentation is not complete and is not easy to follow. The multitude of KB’s are also not complete and not always easy to follow. After the attention I’ve bought to this topic improvements are being made and it will be a lot better in the future. There are a lot of detailed steps that need to be followed. Even my blog posts on this topic (which have now been tested quite a few times) have gone into a lot of detail. One missed step or incorrect step can cause the entire operation to fail. So best not try and do this during a period where you’re sleep deprived. The best advice here is to follow the steps in my blog articles (Refer to the posts listed Updating CA SSL Certificates in vSphere 5) carefully and watch out for updated VMware documentation and KB articles. Always have a backup of previous certs and always try updating certs in a test environment before doing it in production. Make sure when you are requesting and generating your certificates that you select the correct template. You need to use a template is based on the standard Web Server (Assumes Windows CA). Make sure the password in the PFX file is ‘testpassword’ and make sure the certificates are downloaded in PEM base-64 encoded format. 

Missing attributes from a Certificate Signing Request and CA Certificate Templates not configured correctly

Common Mistake 2 and 3 can generally be combined into one area as they are generally around missing attributes or incorrect configuration

For the Certificate Signing Request and the CA Certificate Template they should have the following:

keyUsage             = nonRepudiation, digitalSignature, keyEncipherment, dataEncipherment # digitalSignature, keyEncipherment,  dataEncipherment are Mandatory, nonRepudiation Optional
extendedKeyUsage         = serverAuth, clientAuth # Mandatory
subjectAltName        = DNS:updmgr.homedns.org, DNS:updmgr # Optional, except with SRM

Make sure that you’re Windows CA Certificate template has the option “Allow Key Exchange Only With Key Encryption” selected in its key usage policy. It will not work without this.

You need to verify your certificates when you get them back from your CA to ensure that they include the correct fields and that all fields are populated correctly. The above are in addition to the normal request fields that are necessary for the CSR. You’ll find more information on my previous posts. Refer to the posts listed Updating CA SSL Certificates in vSphere 5 especially regarding vCenter.

Not using text transfer mode when copying cert files to ESX/Linux systems

This is probably one of the biggest catches when updating SSL Certificates, especially when they are generated in a Windows system and then copied to an ESX/ESXi or Linux system. The ASCII text file formats are a little different. If you don’t use text mode file transfer in WinSCP or other secure copy tools you will find additional unwanted characters in your certificate files. This will corrupt the file and render it useless unless corrected. Fortunately this is very easy to correct. Either copy the files again using text mode file transfer, or alternatively follow the process outlined by Maish Saidel-Keesing in his recent post Removing ^M Characters from Files in ESXi at TechnoDrone.

Not having the full CA key chain in the certificate

This common mistake comes thanks to Erik Bussink. It could sort of be covered by option 1 but happens frequently enough to have a separate item.  To ensure you don’t run into this make sure you download the full base-64 encoded certificate from your CA that includes the full key chain. In a Windows CA this is as simple as just downloading the base64 encoded certificate on the final page of the certificate submission workflow. This may also require importing the trusted Root Public Cert using openssl or Java keytool from the command line.

Hopefully by taking the above common mistakes into account, and by following the detailed articles I have posted on the topic you will find changing SSL certs in vSphere  much easier and more successful.

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2012 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.

During the process of working with customers changing their SSL default self-signed certs for CA signed SSL certs in their vSphere environments I found that the order they were changed made a difference. This was also the case when I ran through the same process in my lab environment. Here is the order that I found was the easiest when changing the SSL certs in the vSphere environments I’ve worked with.

Firstly I’d like to start by saying this is what I’ve found the easiest in the environments I’ve worked in. Your mileage may vary. I have tested this with vSphere 5, but it may also be applicable for earlier versions. I’d like to hear from you on your experience and if this has worked for you, or if you used a different order.

1. ESXi Hosts
2. vCenter Server
3. vSphere Web Client
4. Other Components, such as SRM, vCenter Operations Manger, VMware View, vShield etc

If you want a way to fully manage the certificate lifecycle and replace certs automatically then you’ll want to check out vCert Manager – Changing VMware SSL Certs Made Easy. This will completely automate the SSL certificate process in vSphere environments. 

The reason I have found that this order is the easiest is because if you update the ESXi Hosts certs first with trusted CA certs they can be added into vCenter quicker. The hosts will also not become disconnected and require you to reconnect them when you change the vCenter SSL certificates. The reason why I have the vSphere Web Client listed before other components is because it will generally be installed on the same server as vCenter Server. Unless of course you have a very large number of vSphere Web Client users, which which case you will have split it out onto a separate server.

If you can update the SSL Certs on the ESXi Hosts before adding them into vCenter it will save you some time as you won’t have to fix the SSL thumbprints in the vCenter Database, which is due to be fixed in vSphere 5 Update 1  (refer to The Trouble with CA SSL Certificates and ESXi 5).

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com, by Michael Webster +. Copyright © 2012 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.