Utility To Zero Unused Blocks On Disk

Home » CentOS » Utility To Zero Unused Blocks On Disk
CentOS 36 Comments

Is there a utility to zero unused blocks on a disk?

CentOS 6.7/Ext4

I saw zerofree, but I’m not sure it would work on Ext4 or even work on this version of CentOS.

thanks,

-wes

36 thoughts on - Utility To Zero Unused Blocks On Disk

  • Personally, I just do ‘dd if=/dev/zero of=/path/to/zero.img bs=1M; rm -f
    /path/to/zero.img’. It’s inelegant, for sure, but it works (note to run it as a normal user or else be careful of how your system reacts to running out of disk space for a moment).

    fix-it-with-a-hammer-digimer


    Digimer Papers and Projects: https://alteeve.ca/w/
    What if the cure for cancer is trapped in the mind of a person without access to education?

  • Wes James wrote:
    disk, use dban , which surely approaches industry standard for the open source answer.

    Just zeroing random blocks? Why? If you want to wipe a specific file, there’s shred.

    mark

  • Wes didn’t say the reason he wanted to zero unused blocks, but I always do this in kickstart scripts when constructing VM images as the image size is considerably reduced by doing this…

    -Greg

  • DBAN is obsolete. NIST 800-88 for some time now says to use secure erase or enhanced security erase or crypto erase if supported.

    Other options do not erase data in remapped sectors.

    Chris Murphy

  • Chris Murphy wrote:

    dban doesn’t? What F/OSS does “secure erase”? And does it do what dban’s DoD 5220.22-M does?

    mark, overkill

  • the only truly safe way to destroy data on magnetic media is to grind the media up into filings or melt it down in a furnace.

  • John R Pierce wrote:
    Well, no. DeGaussing works, also. Plus, with current storage techniques, I’ve heard that one pass of whatever will make it unreadable. That was why I jokingly referred to DoD 5220.22-M as overkill, since it does 7 passes.

    mark

  • do you even know what NISP Operating Manual 5220.22-M is? One thing it does NOT have is ANY specifications of methods of data erasure (it mentions data erasure in 2 short paragraphs, out of a 140 page book on security).

    The Defense Security Service C&S-M (clearing and sanitization matrix)
    procedures state that magnetic disks will be degaussed or physically destroyed. http://www.oregon.gov/DAS/OP/docs/policy/state/107-009-005_Exhibit_B.pdf

    note that degaussing a hard drive made since the early 80s will erase its servo tracks and render it scrap.

    there is no such thing as secure erasure. the whole silly 3 passes of random data followed by zeroing thing has been debunked numerous times. It MIGHT have worked in the days of MFM disks, when block sparing was an OS function, and the drives just provided a stream of 1s and 0s without recognizing ‘sectors’ (sectoring was done in the MFM disk controller board). With any modern storage device writing a single pass of zeros will do virtually the same thing, and is adequate to remove casual data but by no means good enough for any sort of government mandated security.

  • This definitely does the trick. reallocated bad blocks aside, one path writing zeroes on modern drives is sufficient, according to one nice paper on the subject I remember. Does not comply DoD (and similar) secure data destruction though… As it always is when army is concerned: overkill ;-)

    Valeri

    ++++++++++++++++++++++++++++++++++++++++
    Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
    ++++++++++++++++++++++++++++++++++++++++

  • Without any intent to contradict… This article I found to be very instructive reading:

    https://www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html

    As far as hard drives are concerned, it changed my mind about magnetic media (hard drives): from “you never will be able to securely destroy data” to “one pass of writing zeroes is sufficient for modern drives. Bringing platters over 1000 times deep into hysteresis back and forth is enough to destroy even residual magnetization related to magnetic domain aging… On modern drives though… No, I decided to not spoil it for those who decides to read that article. One thing I learned from there:
    DRAM had more persistent imprint of information that was sitting in it, which appears much harder to destroy than information on hard drive. I
    hope I intrigued you enough to go and read that article.

    Valeri

    ++++++++++++++++++++++++++++++++++++++++
    Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
    ++++++++++++++++++++++++++++++++++++++++

  • well aware of that. 30 years ago a friend and I built a specialized video card for a consulting project…. discovered on the prototype that we could power it down, go home, come back the next day, and power it up, and sufficient image remained in the DRAM to be totally recognizable. sure, it had ‘noise’ bits all over it, but whatever was on the screen the day before was still visible.

  • This is amazing! That is what distinguishes experts from us mortals: what we just learned by reading they saw themselves (sometimes quite a while ago)! On the other hand, why I’m so surprised? This is why they can answer any of our questions on this list in a split second.

    Still, I’m very impressed!

    Valeri

    ++++++++++++++++++++++++++++++++++++++++
    Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
    ++++++++++++++++++++++++++++++++++++++++

  • I unscrew the casing, extract the disk platter(s), slide a very strong magnet over both sides of the platter surface then bend the platter in half.

    How secure is that ?

    I can’t afford a machine that grinds everything into dust particles.

  • Once upon a time, Greg Bailey said:

    For that purpose, use something that can TRIM a VM image, like virt-sparsify.

  • That’s doing the same thing.

    virt-sparsify works by mounting the filesystem, filling it to capacity with zeros, then performing a copy operation which skips over the all-zero blocks, leaving them unallocated in the sparse destination file.

  • a comment on replies to your post.

    i find it interesting that “Subject:” is and you ask for info to

    *zero unused blocks on disk*

    and all the replies for wiping the _entire_ disk.

    why is so hard to understand that _blocks_ does not mean _disk_. ((GBWG))


    peace out.

    If Bill Gates got a dime for every time Windows crashes…
    …oh, wait. He does. THAT explains it!
    -+-
    in a world with out fences, who needs gates.

    CentOS GNU/Linux 6.7

    tc,hago.

    g
    .

  • Once upon a time, Robert Nichols said:

    Well, that’s the slow method that copies the image. There’s also the in-place method, which just does a TRIM that pokes holes in the original image (much faster and uses no extra disk space).

  • this is true.

    also true is that i phrased with;

    and all the replies for

    not,

    with all replies for

    because, yes, i did read you reply with all the rest. ;-)

    fell better? :-P


    peace out.

    If Bill Gates got a dime for every time Windows crashes…
    …oh, wait. He does. THAT explains it!
    -+-
    in a world with out fences, who needs gates.

    CentOS GNU/Linux 6.7

    tc,hago.

    g
    .

  • http://dban.org/download

    That DoD standard is also obsolete per NIST 800-88. There’s zero evidence provided that 2 passes makes any difference compared to 1, let alone doing 7.

    hdparm –security-help

    This takes the form of something like:

    hdparm –user-master u –set-security-pass chickens /dev/sdX
    hdparm –user-master u –security-erase-enhanced chickens /dev/sdX

    The 2nd command doesn’t return until completion. hdparm -I can give an estimate of how long it will take. For HDDs I’ve found it slightly overestimates how long it will take, but is generally pretty close. For SSD’s it can be way off. It says 8 minutes for my SSD, but the command returns in 5 seconds and the SSD spits back all zeros.

    Secure erase is really the only thing to use on SSDs. Writing a pile of zeros just increases wear (minor negative) but also doesn’t actually set the cells to the state required to accept a new write, so you’ve just added a lot more work for the SSD’s garbage collector and wear leveling, so it’s going to be slower than before you did the zeroing. Secure erase on an SSD erases the cells so they’re ready to accept writes.

  • Oops. It’s probably a fairly close approximation to just mkfs.btrfs -f
    (or xfs) the entire block device for the SSD. If the kernel sees it as non-rotational, it’ll issue a whole device trim first, then write out scant amount of metadata (btrfs writes out a tiny amount of metadata at mkfs time, xfs a bit more, ext4 a lot and then even more after mounting).

    For most people this is probably a lot easier than the multistep process using hdparm and secure erase.

  • at least one SSD I had, the vendor told me writing a full pass of zeros on it via dd or whatever would completely reset the garbage collection and effectively defrag it.

  • I have seen a guy in the I.T. department where I work take a hard drive….remove as many stickers from it (to make it porous as possible)
    and then soak it in a “brine” of ammonia, hydrogen peroxide, bleach, and salted water…for like a week or two. Afterwards? total useless junk.

    EGO II

  • I often find harddrives with several 10 mm drill-holes in them, in the electronics disposal room at work.

    I’d say that’d be enough as well. 8-)

  • After seeing Aaron Neville tonight (wow, what a show, 3rd row center seats in a small theater, we were 15′ from the stage), we stopped at The Penny Ice Creamery, and I had a small cone of Meyers Lemon Sorbet.
    very yum, intensely fresh lemon.

  • “Tell It Like It Is”. a great song by a great singer and group.

    i meet the neville brothers in new orleans, la back in mid 60’s just after the second hurricane pushed thru.

    . sorbet is good, but it is not ice cream, it is ice water.

    strawberry and raspberry are my 2 favorite.

  • Actually, while there is some good security to that it’s not for the reason you might suspect. Just taking the platters out is enough, for most drives, unless you have the proper precision jigs necessary to get them properly realigned.

    But a static permanent magnet can leave a lot more than you think;
    degaussing is almost always done with alternating magnetic fields and exponential decay. The exponential decay is the key, and a proper set of passes with a permanent magnet taken at roughly exponentially-decaying heights will do the job (screwdriver demagnetizers have steps in their casing to do this). An AC degausser needs to be kept on while removing from the media to fully degauss (or you end up with more residual magnetism (hysteresis) than can be overcome by the heads).

    Piranha solution does a number on all known magnetic media (and skin), but bringing the media up to transition temperature will also effectively degauss. So melting it down is guaranteed, and fun (but remove the printed circuit board and all other plastics first……).

  • Yes it’d be “defragged” in that it has no file system at all to be fragmented in the first place (easier done with a mkfs). But a huge percent of the available cells on the drive (the portion not overprivisioned) would contain valid data (zeros) as far as the drive firmware is concerned, and those cells storing zeros are not in a state to accept writes. So unless it’s a very good SSD that’s so overprovisioned that it can perform well without the benefit of trim, and some can, this is odd advice. It’s much simpler to just do a full device mkfs, which will do a whole device trim, and if you want you can then use wipefs to remove that filesystem’s signature.

  • At home I just collect up any old disks, memory sticks, credit cards and anything else with storage from the family and keep them until I
    am burning garden rubbish. Once there is a nice orange- or yellow-
    hot firebed in the incinerator I throw them in. Sometimes the aluminium runs out of the bottom, all I find in the ashes is the odd bit of steel. I’d like to see the CIA read that lot!
    —–BEGIN PGP SIGNATURE—

  • Secure erase of an SSD, or any solid state device, is problematic.

    See:
    http://www.techrepublic.com/article/erasing-ssds-security-is-an-issue/

    The CSE requires physical destruction of these devices through pulverisation or incineration. See:
    https://cse-cst.gc.ca/en/system/files/pdf_documents/itsg06-eng.pdf

    The USDOD leaves disposal protocols to the individual commands.

    Essentially, due to the way data is stored on SSDs, it is impossible to access every memory cell during a software driven wipe; no matter how many passes are made. The possibility of significant fragments of residual data remaining is always greater than zero.

    However, if you entirely encrypt an SSD, BEFORE adding any confidential material, then secure destruction is assured by
    ‘forgetting’ the key. But encrypting an SSD after the material is put on it is not sufficient.