NAND flash-based memory has already come to dominate the personal storage market, be it in laptops, tablets, smartphones or cameras. In enterprise applications where storage capacities for a single server are in the terabyte to petabyte ranges, hard disk drives (HDDs) are still widely used due to their lower cost per gigabyte and the availability of higher capacities per drive.
This situation is slowly changing with the increasing affordability of Enterprise SSDs (eSSDs), which is driven by a reduction in the cost per GB. This reduction in cost per GB is being supported by the adoption of new cell technologies such as the shift from SLC (single level cell) to MLC (multi-level cell) NAND flash memory, the shift to smaller processing nodes that enable higher bit densities and therefore greater capacities per unit size of chip, and increased economies of scale as demand grows.
eSSDs were initially adopted in cache applications and transaction processing systems that benefited greatly from the higher random access performance of NAND flash memory storage compared to HDDs. However, due to the high number of write/erase cycles in such applications (often greater than 10 entire drive writes per day [10 DWPD]), rather expensive SLC (€/GB) flash technologies were required. This limited the capacity of the drives used to just a few 100GB per device, which was fine for cache applications and transaction processing, but not for larger volume of data.
Today, applications requiring drives that can withstand 10 or more DWPD use eSSDs with specially selected MLC NAND flash memory, a high-level of over-provisioning and optimized error correction technologies.
Higher Capacity Futures
As we move into 2015, content delivery networks that offer digital music, video and games for download and streaming applications are also demanding higher data read performance. Because the content is not often updated, the frequency of write/erase cycles is lower.
To meet the needs for these customers, eSSD manufacturers have developed drives based on higher bit density NAND flash memories that can withstand up to 3 DWPD. In comparison with 10 DWPD drives, these enterprise SSDs are available at lower price per GB. In addition, they incorporate higher levels of overprovisioning compared to client SSDs, complex wear-leveling algorithms, robust error correction technologies and power loss protection.
This segment will drive a large portion of the growth in demand for eSSDs, with adoption of eSSDs to store terabytes of data. It is expected that applications with moderate write workloads and those where random write workloads can be cached into larger sequential blocks before transferal to the SSD will be the next segments to adopt large capacity SSDs. These applications include virtualised environments such as Virtual Desktop Infrastructures (VDI) and virtual servers.
The transition of active databases to eSSD-based systems is less obvious. While the high random in/out operations per second (IOPS) performance of eSSDs makes them extremely attractive for active databases and online transaction protocol (OLTP) applications, these applications have the highest workload. Such workloads have the potential to stress the write endurance of the NAND flash memory cells to their limits, and therefore these applications require either “high-DWPD” eSSDs or multiple 15,000rpm HDDs running in a RAID configuration. The ultimate decision comes down to a trade-off between cost, performance and implementation risk.
As the cost per gigabyte of NAND flash memory falls, the trend will increasingly favour flash-based memory options. If IOPS/random performance was the only optimisation criteria, a RAID1 set up of two high-end 12Gbit/s SAS eSSDs with 400k IOPS could replace up to 2000 spindles of 10k/15krpm drives which would be required to achieve the same IOPS level. In such extreme case, the high price of the 10+ DWPD eSSD would still make economic sense.
Interfacing With The Future
The majority of consumer HDDs and SSDs use 6Gbit/s SATA interfaces. On top of the client SSD architecture, an Enterprise SATA SSD usually comes with a defined DWPD (typically 1 or 3 DWPD), data protection for unexpected power loss and warranty terms that match enterprise operation. Enterprise SATA SSDs achieve higher DWPD ratings compared to client SSDs by incorporating higher levels of over-provisioning and firmware optimisations to meet the requirements of operation in an enterprise environment.
While large capacity eSSD-based storage architectures will stay to a certain extent with SATA-interfaces during 2015, it is expected that increasing numbers of systems that house active data sets will migrate to faster interfaces that do not throttle performance.
In storage systems, 12Gbit/s SAS interfaces can enable up to four times the internal performance capabilities of the flash memory than SATA interfaces. The faster SAS interface and the dual data path capability provides performance gains and failure protection potential that offset the price differential with comparable SATA-based products for many applications. Indeed, manufacturers have started to launch low-DWPD SAS eSSDs to maximise the IOPS performance that storage systems can deliver.
It is worth noting, that NAND flash-based storage can even outperform the SAS 12Gbit/s interface, and this will lead the increasing adoption of PCI Express (PCIe)-based eSSDs. PCIe interfaces support data transfer rates of 8Gbit/s per lane, with multiple lanes providing almost unlimited scalability of total data-rate.
All-flash arrays have used plug-in cards based on the PCIe bus for some time, but only for a limited number of applications. The reasons for this include the lack of a hot-swap capability for such cards, and the rather complex integration process that requires special proprietary drivers to enable the use the flash memory on the PCIe bus.
The recently defined and implemented NVM-e (Non-volatile-memory express) interface uses the PCIe protocol, but with an interface compatible with the SAS/SATA disk interface. It adds more physical lanes – up to 4x PCIe lanes – on a 2.5-inch form factor device. NVM-e features support for hot-swap removal of drives that enables failed drives to be removed without powering down the server. However, the number of devices usable in a system is limited to the number of PCI Express lanes on the motherboard.
The missing piece of the ecosystem of controller and expander modules will limit the usage of PCIe-based eSSDs to a few devices integrated into the server. Wide usage of PCIe based SSDs in larger storage arrays will require an ecosystem of controller and expander modules similar to that established for SATA/SAS. This will take more years to establish, so PCIe/NVM-e based eSSD adoption will start in 2015, it will be limited to a few devices in compute servers, while larger storage arrays will remain SATA/SAS based for the next few years.
Sales of eSSDs will continue to grow through 2015 and beyond. As the cost per GB continues to fall, adoption will intensify and widen into new applications across the enterprise, while the adoption of new technologies and interfaces will further enhance the performance boost that eSSDs provide. To be successful in the competitive enterprise market, eSSD manufacturers will need full control of design and development of the next generation controllers and NAND flash memory cells.