Sanity News

The Bottleneck Moves to the Network - Why NVMe Over Fabric May Soon Be in Your Data Center

Wednesday, October 04, 2017

Seismic shifts and advances in technology are occurring at what seems like an accelerated pace across a pervasive spectrum of data center infrastructure including servers, storage and networking.

Recent announcements by Intel, AMD, and NVIDIA are evidence of the continued exponential increase in raw processing power and throughput of that is fueling technology breakthroughs in Artificial Intelligence, machine learning, and real-time big data analytics across all disciplines, industries and economies.

The Internet of Things (IoT) which encompasses the networked connectivity of physical devices with intelligent sensors and software to integrate the physical world into computer-based systems. Experts are predicting that 2020, there will be more than 30 billion objects interoperating within the existing Internet infrastructure, facilitated in part by the coming 5G wireless technology.

This general class of cyber-physical systems is what many pundits are calling the Fourth Industrial Revolution. Adding computer-based intelligence turns everyday devices and systems into “smart objects” which will user in automation and enhanced applications including smart clothing, smart homes, smart cars, smart grids, smart cities, and smart robotics. It is easy to envision exciting innovations in the way we live (e.g. fitness, health monitoring), work (computer-aided everything), commute (autonomous vehicles and smart cars), consume entertainment (Virtual Reality), and even interact with the physical world (Augmented Reality).

What does all this mean for enterprise IT infrastructure? It means for companies to survive and thrive in the digital age, they need to rapidly modernize their data centers and increase their company’s agility, be more competitive, respond faster and better, and leverage AI, machine learning and real-time analytics to transform their legacy businesses before they are disrupted (e.g. the “Amazon effect”) and/or to pro-actively disrupt their competitors and take market share.

What has become abundantly clear is that performance enhancements in CPU, memory, and NVMe SSDs have moved the bottleneck to the network. Let’s compare in the figure below (using a sprinter analogy) of the speed of a 15,000 RPM hard disk drive to a standard 12 Gigabit SAS SSD and then to a PCIe based NVMe SSD.

Now let’s consider Dell’s new R740xd servers with Intel Skylake processors that can accommodate up to 24 hot-pluggable NVMe SSDs. In the figure below, note that it will require 2 x 100 Gigabit Ethernet links to barely keep up. The network will be the new choke point or bottleneck!

And what will be the best way for one server to quickly access data on another server? The answer is NVMe over Fabrics, which is effectively utilizing the NVMe interconnect standard plus high-speed RDMA (Remote Direct Memory Access). RDMA is the ability to directly access the memory of one computer into that of another without involving the other computer’s operating system. This permits high-throughput, low-latency networking and is commonly used in massively parallel computing clusters. Note that the actual protocol be Fibre Channel, InfiniBand, RoCE (RDMA over Converged Ethernet), IWARP, or GENZ (a proposed next generation fabric).

Where will we see NVMe and NVMeOF? The answer is EVERYWHERE. Expect to see the major server vendors adopt both NVMe and NVMeOF. You will also see all flash array vendors and even hybrid flash array vendors adopt this technology.

Case in point, on August 21st, Tegile Systems launched their IntelliFlash N Series All-NVMe Storage which is scheduled to be available this fall. Check out the full press release here for more information on Tegile’s latest all NVMe offering. This is the first unified All-NVMe array on the market to deliver full enterprise functionality, including robust data management capabilities. Tegile has consistently earned top rankings in the DCIG Flash Array Buyer’s Guide, and a “Visionary” in the Gartner Group Magic Quadrant for Solid State Arrays.

In a deeper dive, “The Business Value of NVMe” by Neuralytix provides the following guidance and conclusion, “NVMe and NVMe over Fabric can help companies improve productivity, increase revenue, be more competitive, transact more…respond better…and scale in a pay-as-you-grow model.” Get the full white paper here.

With today’s pace of change in technology, there are several moving parts and a multitude of considerations to keep in mind when setting your IT directives and strategies. As students of the industry, Sanity Solutions’ certified team has a constant finger on the pulse of innovation to make your next IT transformation a success. For more information on how Sanity can help your business, contact us at