Persistent Memory Was A Popular Topic At This Week's LinuxCon Europe

[starshipeleven]

Very interresting, this persistent memory can make systems evolve in a great way. Does anyone have informations about the cost / GB of this future technology?

You need to state the technology name, as "persistent memory" is too generic.

AFAIK only MRAM is the tech that has some promise, and has been developed for a long time until now. For now they are still making relatively small capacity chips for embedded (some are also in satellites) as they are building steam.

[starshipeleven]

yes, if "technology" is intel 3d xpoint. no, if "technology" is persistent memory

Intel 3D xpoint isn't anywhere near good enough to replace DRAM.

[starshipeleven]

They early 2000s' Palm PDAs would like to say "hi" to you.

Persistent memory in this context means that you don't have anymore a distinction between RAM and permanent storage, which is a bit different from just battery-backing a tiny amount of RAM while most data is still on a slow permanent storage medium.

[pal666]

Intel 3D xpoint isn't anywhere near good enough to replace DRAM.

i didn't say that, i aggreed that it is 'brand new' and 'does not need battery'

[Ferdinand]

Why don't smartphones use this technology? I have the feeling that most of the wait time and frame rate skips are from storage in phones. And my current phone has 4GB so why not use 1GB as a buffer before it is written to storage. Maybe turn it off when the battery reaches 5%?

[starshipeleven]

Why don't smartphones use this technology? I have the feeling that most of the wait time and frame rate skips are from storage in phones. And my current phone has 4GB so why not use 1GB as a buffer before it is written to storage. Maybe turn it off when the battery reaches 5%?

If you mean MRAM or similar technology, the higher capacity chips are less than 30 MB or so, not suitable for mobile (yet).

If you mean using ram as cache, all embedded devices do that already. Android, iOS and friends live in RAM and do their best to not write/read from storage unless it is absolutely necessary, if the one making the firmware provided the right settings anyway.

There are some tunables for that that may help (if you are root on your phone), like readahead (when reading reads more than necessary and stores into RAM as cache to prevent having to read again for files near the ones you just read) or dirty writes (longer time the device is writing to RAM before flushing to permanent storage).

[Ferdinand]

If you mean using ram as cache, all embedded devices do that already. Android, iOS and friends live in RAM and do their best to not write/read from storage unless it is absolutely necessary, if the one making the firmware provided the right settings anyway.

I don't think they do. I have a Samsung s7 with 4GB memory of LPDDR4-1866 34GB/s. The soc is comparable with a 1Ghz Intel Core cpu. So if I install a 60MB application it should take 1 second to install it to memory which it can then write to storage in the background while not making me wait. It should also take 10 seconds to boot the whole phone but it takes 35 seconds. This phone flies most of the time but it has these weird moments where it somehow doesn't have enough power to keep up. I blame storage for that.

[starshipeleven]

I don't think they do. I have a Samsung s7 with 4GB memory of LPDDR4-1866 34GB/s. The soc is comparable with a 1Ghz Intel Core cpu. So if I install a 60MB application it should take 1 second to install it to memory which it can then write to storage in the background while not making me wait. It should also take 10 seconds to boot the whole phone but it takes 35 seconds. This phone flies most of the time but it has these weird moments where it somehow doesn't have enough power to keep up. I blame storage for that.

That's a case of "people have not set the caches correctly", probably. Or it might be another case of "Samsung manages to have even flagships lag because of their own customizations like TouchWiz interface".

If you can root it you can install apps like this https://play.google.com/store/apps/d....kerneladiutor that allow you to set the values I was talking about to higher values so it uses larger amounts of cache (or stops killing cached/running apps when you have like 128MB of ram still free).

To get better boot times you need to disable/remove all useless shovelware apps Samsung packs in their devices, as they are all starting on boot, and everything comes from storage when you boot the device.

[starshipeleven]

EDIT: by "That's a case of "people have not set the caches correctly", probably."
I mean that Samsung firmware developers did that wrong.

[name99]

They early 2000s' Palm PDAs would like to say "hi" to you.

Note: Back then some other company had managed to land a patent on using flash memory as storage medium on PDAs. Instead of paying the patent, Palm went for using only RAM + batteries in their PDAs. Part of this memory is used as actual RAM for the programms, the rest is used as a form of persistent storage (Thanks to the battery) on which databases where used to store the data.

In other words: it's been 20 years since mass produced computers have been using persistent RAM as storage.

Even back then, this permitted extremely fast application swtiching, searching of storage, etc.

I have no idea what Palm were doing. But most of the commenters here (including you) don't seem to appreciate the technical points that are stake here.
The issue is not that NV-RAM is a faster storage medium, just like SSDs were faster than HDs.
The issue is that because NV-RAM is
(a) byte addressable (or perhaps cache-line addressable)
(b) RAM (duh) as opposed to IO
you can read and write to it without going through the OS mechanisms of files, an IO stack, interrupts, etc.

So what does that mean? The first, trivial, thing it means is that, if you still want to use a file system (and most people will, at least initially) your file system will now be a user-space library, and protection will be implemented by the standard TLB page protection mechanisms (with all that may or may not mean for the finer granularity security that some file systems provide). But more important, robust file systems rely on data being persisted to storage in a very specific order. For IO that order is imposed by sticking fsync()s or something similar between specific writes. With current instruction sets, the equivalent instructions don't really exist or are not optimal. What you need is additional instructions that can be inserted between writes that will flush out previous writes to the persistent RAM. (Or provide other, smarter, ordering mechanism; but flushing is the simplest scheme so will probably be used first). Intel only recently added such instructions, likewise ARM in the ARMv8.2A ISA.

The second thing it means is that existing file systems really should be altered to exploit the new capabilities NV-RAM offers. In particular not having to funnel everything through chunk-sized granules (chunks being 4 or 16kB or so) allows for a different set of tradeoffs in the design of the OS, buying you some percentage higher performance. MS has done substantial work in this, though, being MS, one expects that research will probably stay buried in Microsoft Research for twenty years before it finally, maybe, makes it into a replacement for NTFS.

The third thing it means is that, over time, for at least some purposes, people will consider alternative storage models to files. Some devices did this in the past --- Newton "soups" were a kinda non-filed-based storage system, and various cloud storage schemes are likewise not file based (Apple offers key-value stores for apps using iCloud, and I'm sure so do various other APIs).

But ALL these cool new ideas require something to enforce the ordering of writes out to the NV-RAM. Without that in place, you're playing amateur hour. That's the part that's missing right now from the actual CPUs in the field. And that's the part that requires careful new additions to every OS, whether Linux, Apple, Google, or MS.
You can't just treat this stuff like a RAM-disk, because RAM-disks didn't persist if the CPU crashed/lost power half-way through a storage operation. NV-RAM, if it sees the stores in the wrong order and power is lost part way through, will be basically a scrambled file system, like we used to see back in the 80s before the PC OS vendors took write ordering seriously.