MemVerge is Amazing

MemVerge makes it possible to use Intel’s Optane persistent memory tech with existing applications without rewriting them. This is excellent.

Change is hard, and people don’t like to do it.

I like to say that the first 80% of writing software is putting bugs in and the second 80% is taking them out again. It’s complex and time consuming and expecting enterprises to first rewrite all their applications before they can use your shiny new tech is unrealistic.

Applications will get rewritten over time, sure. Some apps will simply get abandoned, some will get replaced with completely new ways of doing things, and the big squishy middle will be a combination of the two. No technology ever really goes away, which is why we still have mainframes, tape, and FibreChannel.

But you can load COBOL from an SSD into DDR4 memory and run it on an Intel Xeon chip, none of which existed when COBOL was invented. One of the beautiful things about computers is they are basically rocks we tricked into doing maths for us. With the right abstractions, we can do the same old maths on newer, shinier rocks.

Running a program with MemVerge is as simple as calling it via the mm command. This places the memory of the process under the control of the MemVerge memory machine, and makes its management tools available. Tools like: snapshots of live memory.

The MemVerge architecture.

MemVerge can take snapshots of the memory image of the process. This is really handy for a rapid restore of a large in-memory database, which is a big selling feature for the kinds of people with lots of money to spend on shiny new tech like Optane.

It also means you can do fancy tricks like: taking a memory snapshot of a process and restoring it on another server. Maybe you want a rapid restore because a node in a cluster went down, or maybe you want to do memory forensics to figure out where a memory leak is in the code, or watch malware trying to evade your anti-malware tools.

Yes, There’s More

Because of the way MemVerge works, it operates on memory, not just Optane. Computers already use memory, so a bunch of the MemVerge features will work in a computer that doesn’t even have Optane!

MemVerge provided a demo of their software running in an AWS EC2 instance in us-west-1 that didn’t even have Optane installed. This was aimed at showing that the MemVerge Memory Machine abstractions didn’t impact performance very much, but it handily showed just how useful a drop-in component like this can be. I can get a bunch of the benefits of MemVerge techniques before I even deploy a single piece of new hardware!

I was very surprised by how often MemVerge said “yes, you can do that” during their presentation when I was expecting them to say it was a roadmap item or just a good idea they’d consider looking at.

There are all kinds of interesting possibilities from what MemVerge’s software can do all on its own, well beyond just making Optane accessible for applications that already exist. Teleporting processes around a cluster to do near-instant load-balancing is just one. The potential for doing malware analysis on snapshots of a live sample as it executes are another.