This is part of a series of posts about Storage Field Day 9.
Plexistor sell something they want to call “Software Defined Memory” which is an odd term that feels like trying to jump on the bandwagon of calling things software-defined. I guess the thinking is that it’s a variation of software-defined storage, but applying to memory.
Plexistor sell some software that acts as an abstraction layer for memory and storage, and wants to make them both usable with the same interface. This blog post at Plexistor does an ok job of explaining why.
I’ve written before about the implications of persistent memory, and it looks like Plexistor wants to provide the software abstraction layer to make it all happen. Instead of having to learn how to allocate persistent or non-persistent memory inside your application, Plexistor can manage it for you, apparently.
Right now, the OS sort of already does this, but not very well. Modern systems map physical memory (the DIMM sticks you know and love) into ‘virtual memory’ which can be larger than the physical memory. If running applications need more memory than what’s available in physical memory, some of the physical memory can be freed up by moving idle workloads into swap space, paging it out to disk.
Generally you want to avoid this, because if you need that application data again, it has to be swapped back into active memory, and something else swapped out. Lots of paging to disk is generally a bad thing for performance, because I/O is orders of magnitude slower than the memory channel. Which is why all this memory channel storage type stuff is coming along now. People want the low-latency, high-bandwidth of memory. Paging out to block storage defeats the purpose.
The swapped memory isn’t persistent, either. Swap space isn’t preserved between reboots, so if you lose power to the server, the swap space will be overwritten at reboot (and on many modern systems the swap space is encrypted so you can’t make off with active memory just by turning off the system and nicking the drive).
Plexistor want to appeal to people who want to use in-memory applications, like Cassandra, Spark, HBase (and I’d guess in the future stuff like SAP HANA) for fast analysis of lots of data. This is a pretty extreme niche, at least for now, but the people who want/need that kind of performance and scale have deep pockets, so it’s a high margin market to go after.
It’s also right out there on the bleeding edge of new tech, so it’ll be interesting to hear how Plexistor plans to handle the persistence part of things without making people rewrite their apps to tell the difference between persistent and non-persistent memory. They have something called a multi-tier NVM (Non-Volatile Memory) filesystem that does auto-tiering, apparently, but my experience with storage systems doing auto-tiering isn’t that stellar. Auto-tiering is a tricky problem, particularly with large and non-repetitive datasets like those likely to be used by Plexistor’s target customers.
It looks like a really early play into a market that is only just starting to form. Getting the timing right is tough. The product has to be mature enough to be useful and not too early to market so there’s no one there with money. They also need to beware those who follow just behind them, learning from the early mistakes and coming into a more mature market with a potentially more mature product.
If Plexistor can get it right, they might get to define what this brand new market actually is, which would be pretty exciting.