TFD10 Prep: Diablo Technologies

This is part of my series of posts for Tech Field Day 10.

Diablo have had an interesting time since I last looked at them in detail, a bit over a year ago. Diablo was ruled the prevailing party in the lawsuit filed against it by Netlist in September 2015, drawing to a close the question about whether Diablo pinched technology from Netlist for its Memory Channel Storage.

Diablo have also added servers from Huawei and SuperMicro to the list of supported platforms to the fairly limited range of Lenovo servers that were slated last time I looked.

Diablo also presented something called Memory1 at the Flash Memory Summit towards the end of last year (that I didn’t go to, but kinda wish I had now).

I’m not clear on what the differences are between Memory1 and Memory Channel Storage (MCS), as they’re both listed as products on Diablo’s website. Memory1 looks to me like a branded, packaged version of the MCS technology.

What It’s For

I’m struggling a little with the way Diablo are marketing their stuff. The difference between Memory Channel Storage and Memory1 isn’t clear enough from the materials on their website. The presence of ‘Memory’ in the name for both is part of the problem. With some digging (and asking a few questions on Twitter and elsewhere) I think I’ve figured out the main difference:

• Memory1 looks like memory.
• MCS looks like storage.

Both have flash in them, and both plug into the DIMM (DDR4) slots on the motherboard. But the way they present to the operating system is where the difference comes in, and is really what important.

What MCS Is For

MCS makes flash storage faster. It’s basically flash storage connected to the memory bus, instead of over an I/O channel. This means it’s faster to access, because it’s closer to the CPU, and it’s also more predictable because you reduce the number of possible contention issues.

Specifically, it overcomes the limitations of the PCIe bus when you start pushing a lot of writes, or even lots of reads. You can hit the limits of what a limited lane PCI controller can handle because of the way PCIe is architected. The memory bus is designed to handle lots of parallel I/O movement to and from a CPU in a very predictable way.

MCS is not for your average use case. This stuff is for very high-performance workloads, like high-frequency trading or big data analytics things, where you need to be able to access lots of storage very quickly.

From our discussions with Diablo and Sandisk last time, the flash sits behind a DDR3 to SATA bridge, which actually slows the access path down a bit. I’ll be interested to see what progress has been made towards getting that bridge thing out of the way so the SATA part isn’t slowing things down.

What Memory1 Is For

Memory1 gives you lots more memory cheaper than DRAM.

There are plenty of applications that want to run purely in memory. SAP HANA is one you’ve probably heard of, but there are others (in-memory databases from the major vendors, various startups, and custom applications).

In fact, all programs process data in memory. The two majors reason we have storage is that memory isn’t a) persistent, and b) cheap (relatively speaking). So we compromise, loading data from cheaper, persistent, and voluminous permanent storage into memory for processing, and then writing it back out again so we can look at it later.

Memory1 appears to behave a bit like a hybrid flash/HDD storage array, only with DRAM playing the part of the flash cache layer, and the flash being the slower backing storage. Memory1 looks like a dirty great pool of memory, only the vast bulk of it is actually flash. When a CPU needs to fetch stuff from memory, Memory1 does a bunch of fancy tap-dancing to make the big pool of slow (relative to DRAM) flash behave a lot more like memory, and I would guess it comes in somewhere between DRAM and flash disk in performance.

My Concerns

The big difference between flash and memory, for me, is that flash is persistent while memory is not. I’m not seeing anything here that takes advantage of this qualitative difference.

I’d say part of this is because of a need to sell a product. It’s hard to sell people on something that is completely different to the way they do things today. You need a way to explain it to people in terms they understand. Storage, only faster is one way. More memory, only cheaper (and slower) is another.

I hope to understand where Diablo are going with this a bit more when we talk at TFD10. I hope it involves being able to access persistent memory from my programming language using some sort of special device path or API call.