SQLite 3.42 Released With "Secure Delete" Command

[phoronix]

Phoronix: SQLite 3.42 Released With "Secure Delete" Command

SQLite 3.42 is now available as the newest update to this widely-used, embed-friendly SQL database option that is used by countless applications and other software for lightweight and speedy data storage purposes...

https://www.phoronix.com/news/SQLite-3.42

[milkylainen]

I like how they made non-cheating delete operations sound spiffy.
"Normal delete? NAH! Secure delete!"

[DeepDayze]

This sounds like a great idea to ensure that deleted data cannot be recovered, especially within secure environments. However using this option will mean that users will have to be careful with deletions, but programs can perhaps mark deleted records with a soft delete flag and then later can be hard deleted if not necessary.

[qlum]

I like how they made non-cheating delete operations sound spiffy.
"Normal delete? NAH! Secure delete!"

Just because you delete data you no longer need doesn't mean you want to deal with the performance cost of completely wiping the data, itjust is not relevant most of the time. The name secure delete is pretty clear on the matter, you should only use it if you care about the data being actually erased.

[Jakobson]

This sounds like a great idea to ensure that deleted data cannot be recovered, especially within secure environments. However using this option will mean that users will have to be careful with deletions, but programs can perhaps mark deleted records with a soft delete flag and then later can be hard deleted if not necessary.

If a secure program unintentionally deletes data then the issue is totally somewhere else than in the delete function of SQLite.

[fazalmajid]

IIRC SQLite was used to detect NSO Pegasus infestation in iPhones. There is a SQLite DB that tracks process execution and the malware tried to expunge traces by deleting from those databases, but did not perform a VACUUM and so there were still traces.

https://www.amnesty.org/en/latest/research/2021/07/forensic-methodology-report-how-to-catch-nso-groups-pegasus/

[stormcrow]

This sounds like a great idea to ensure that deleted data cannot be recovered, especially within secure environments. However using this option will mean that users will have to be careful with deletions, but programs can perhaps mark deleted records with a soft delete flag and then later can be hard deleted if not necessary.

It should go without saying that "user will have to be careful with deletions". I know the practical reality is that most people are not and people make mistakes. However, the average person, even a lot of advanced users (even programmers!) who probably should know better, don't know that "delete" and "move to trash" does not and never has in most OSes and file systems actually destroyed data. Because of performance reasons, delete only removes the reference pointer to the first sector(s) of the on disk meta/data information chain that the file concept abstracts. Trash cans are even above this misnomer of an abstraction. The trash can only moves a trashed file into a special hidden folder. Even "empty trash" does not, and never has on nearly any FS actually deleted that data from the storage device. It usually takes a special program to go in and manually zero out the data on a mechanical hard drive to make sure it's gone. SSDs are a little more complicated in that you can delete, but the data remains till next TRIM cycle - assuming the garbage collection isn't broken. This ignores the problem with reallocation and wear leveling, of course.

Therefore, since a lot of programs use SQLite to handle file level operations, even ones that don't actually use a database per se, it's a good idea that SQLite introduces the secure delete as a separate concept from traditional file delete operation. One anticipates a certain backward compatibility between the expected result of a 50 year old historical term "delete" regardless of what naive people think it means, and the newer operation that scrubs the data in whatever way the OS and filesystem allows. Keep in mind that some filesystems or OSes don't actually allow scrubbing data because they have a transactional snapshot abstraction that allows historical data operations auditing for certain regulatory regimes. The original data remains in a snapshot and will remain indefinitely depending on the underlying FS mechanisms, and there may or may not be any easy way of clearing those snapshots even with this SQLite feature.

[kobblestown]

I like how they made non-cheating delete operations sound spiffy.
"Normal delete? NAH! Secure delete!"

When you perform a "normal delete" on a file, its content does not disappear. It's simply no longer being pointed to. Hence the existence of undelete software.

The use of TRIM changes that but only for SSDs.

[mb_q]

When you perform a "normal delete" on a file, its content does not disappear. It's simply no longer being pointed to. Hence the existence of undelete software.

The use of TRIM changes that but only for SSDs.

Trim is also logical, but on a level of the SSD controller -- the data is still on the flash, closer in the reuse queue but most likely still far from its head. Similarly, there are COW filesystems, backup daemons, etc.; the only way to make something reliably deletable is to write encrypted and throw the key far from the system, onto some medium which does support destruction.