There is a lot of talk from All Flash vendors and vendors of different types of Hyperconverged (combining storage and server with a hypervisor in one package) system vendors also about which type of system is better and provides better data reduction. Unfortunately in most cases their arguments are somewhat simplified from reality.

Some say you need all flash or data reduction doesn’t work, others say you need a hardware card else data reduction doesn’t work, some say you need a traditional SAN. You can get good data reduction from multiple techniques and multiple different systems, so who is right? The truth is that data reduction savings are based more on the type of data being stored, than it is on how you do the data reduction. If you are storing already compressed image data or encrypted database systems, your data reduction savings will be very small if any. If you are storing easily dedupable data like VDI images, or compressible data, like text, then your savings will be good.

Some vendors try to manipulate the numbers to say snapshots are dedupe, or that data provisioned from templates are dedupe, both can be misleading. Taking a snapshot does not reduce data (and is not a backup), however provisioning from template using smart metadata operations does reduce data, but it’s not dedupe. Dedupe only applies when a pattern of new incoming data is recognised and then not needed because the pattern has already been stored in the past. Dedupe works great on some types of application, and not well on others, just like compression works well on some, and not on others.

The worst argument of them all is that you must have all flash in order to get good data reduction or it will impact performance, this is simply not true, unless your vendor doesn’t know how to design their platform. To show that Hybrid software defined systems, which include SSD and HDD, can give good data reduction and there is no problem with performance lets have a look at the results from some real customer environments.

All of the results I’m about to share have been sent into me or my colleagues at Nutanix from environments that are getting these results on standard hybrid systems across many different use cases. Given that I have access to the data about Nutanix this is what I’m using. Other systems may show similar or different results. Based on your particular system the results may vary.

Here is an example a server workload that contains a lot of easily compressible data and therefore shows good data reduction:

In this case the data was from an Exchange JetStress test. JetStress data is easily compressible and for this reason any system that tests with compression turned on will provide invalid results. So when testing with JetStress you should actually turn off compression. If your storage system can’t turn off compression, then you won’t see real world results (not that JetStress is real world anyway).

This next result is from an Oracle RAC Database. The database contains transactional data, like TPCC would. As a result you get good data reduction. But not as good as if it were just text.

The next example is from a Management Cluster for a VMware environment. It contains vCenter, vCenter Database, VMware Management Tools, Microsoft AD, DNS, and some backup copies of other systems, such as SQL Server and PostgreSQL databases.

A lot of the binary data is already compressed in the above example, so you only get some data reduction savings.

What about real Exchange systems in production environments? Here is an example from a Nutanix customer running an Exchange environment with more than 20K mailboxes. The email data is of all types.

The thing with email data is that one environment can be completely different to another. Here is another example of email data, this time it is from the Enron Email database. Enron is the company that very publicly collapsed after it had a lot of off balance sheet entities and debt. The email database was made public as part of the court proceedings. So it’s a great test case for data reduction technologies.

So in this case we are using compression. Compression is a good choice if data is unlikely to be very common. But you can turn on Dedupe, Compression and other data reduction techniques, such as Erasure Coding all at the same time to achieve the best reduction possible and the platform should work out the best technique to use based on the actual data.

Some types of workload yield extremely good reduction savings from Dedupe, including VDI environments, both full clone and linked clone. Full clone environments yield the best reduction savings from Dedupe, as almost all of the data is duplicated. You can use both DeDupe and compression to get the best possible savings for the data. But here is a couple of different examples from VDI environments.

The above two examples are from two different VDI environments with slightly different workloads. But you can clearly see that the combination of compression and deduplication technology give very impressive savings. But these environments are largely linked clone environments. Not full clone desktops. Here is an example of the data reduction using dedupe in a full clone VDI environment:

This clearly shows that if all of your data is common between the different workloads that you can expect to get extremely good data reduction savings and therefore use very little physical space for those workloads.

So far we have covered a few different types of workloads, but at relatively small scale. But what happens when we scale up to larger environments and more enterprise type workloads?

The above image is from a slightly larger environment that has 10 hosts serving 70 large VM’s doing a lot of large IO. In this case the data reduction is deliver savings of 175TB. But I think we can do better than that. Let’s take the scale up a bit to a slightly larger production environment.

The above example is from a large Nutanix environment with mixed workloads including SQL Server and archive data. In this case the combined data reduction is delivering savings of over 570TB.

The above example is from 2 clusters that each consist of 32 Nutanix nodes. The workloads are general server virtualization with Microsoft applications in one cluster and Linux based applications in the other.

In this above example the cluster consists of 4 all flash Nutanix nodes running on Lenovo HX appliances. The data is a mix of Oracle and SQL Server databases, with a few VDI desktops.

The above image is for a large 26 node hybrid cluster that contains mixed node types and among other applications a 20+TB database.

Final Word

All Flash is not mandatory for data reduction. As you can see from the above you do not need All Flash to get great data reduction savings and acceptable performance. You can get great reduction from hybrid storage environments, and if you choose to go to All Flash you should expect to get the same good data reduction savings. But that decision to go hybrid or All Flash is up to you. Both types of environment can deliver the same reduction savings because it is based on the type of data being stored.

You don’t need special hardware to get great data reduction and performance. All of the above results are from software only and do not require any special hardware. The performance and reduction is dependent on the type of data. Public clouds don’t rely on special hardware, why should your private cloud?

The biggest factor determining data reduction results is the type of data being stored. Yes there are differences in data reduction techniques across various vendors, but the biggest factor that determines data reduction results is the type of data being stored. You need to look through the invalid comparisons using snapshots and calling it dedupe when it’s not. You will get very poor results from data types that don’t reduce well, such as encrypted data, data that is already compressed, images etc. You will get excellent results from data that reduces very well, such as many common images of the same OS or application, data stored in documents, text and databases that has text type data. Because of this it is important that your systems use multiple techniques of data reduction so that you can use the best reduction based on the actual type of data.

Some benchmark tests will provide invalid results if compression is enabled. If your system doesn’t allow you to turn off compression and dedupe you may get invalid results during some types of benchmark testing, such as with Exchange JetStress. This means you can’t use these results to determine how production environments might behave.

Snapshots and clones are not included in the above results! Snapshots and clones, which use metadata operations only, do not increase data and therefore are not included in data reduction results and reports on the Nutanix platform. They also do not increase storage usage, as they don’t create any net new data until new writes are received. Therefore you can take as many clones and snapshots as you like and it doesn’t impact your useable storage. Other vendors have chosen to report this as part of their data reduction savings, however Nutanix only reports savings from actual compression or data deduplication resulting from real pattern recognition.

To sum all of this up. Your data reduction will be based on the type of data that is stored more than anything else. Due to this your milage may vary. Use real world data and real world workloads in any pilots so you can see what real data reduction might be for your real workloads. Big thanks to all of the customers that submitted images with their permission to use them anonymously in this article.

—

This post first appeared on the Long White Virtual Clouds blog at longwhiteclouds.com. By Michael Webster +. Copyright © 2012 – 2016 – IT Solutions 2000 Ltd and Michael Webster +. All rights reserved. Not to be reproduced for commercial purposes without written permission.