Imaging A Drive With Dd

Home » CentOS » Imaging A Drive With Dd
CentOS 16 Comments

I am building a mailserver and with all the steps, I want to image the drive at various ‘checkpoints’ so I can go back and redo from a particular point. The image is currently only 4GB on a 120GB drive.
Fdisk reports:

Disk /dev/sdb: 111.8 GiB, 120034124288 bytes, 234441649 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x0000c89d

Device Boot Start End Sectors Size Id Type
/dev/sdb1 2048 1026047 1024000 500M 83 Linux
/dev/sdb2 1026048 2074623 1048576 512M 82 Linux swap / Solaris
/dev/sdb3 2074624 6268927 4194304 2G 83 Linux

and parted:

Model: Kingston SNA-DC/U (scsi)
Disk /dev/sdb: 120GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos Disk Flags:

Number Start End Size Type File system Flags
1 1049kB 525MB 524MB primary ext3
2 525MB 1062MB 537MB primary linux-swap(v1)
3 1062MB 3210MB 2147MB primary ext4

what dd params work?

dd if=/dev/sdb of=os.img bs=1M count210

?

thanks

16 thoughts on - Imaging A Drive With Dd

  • That looks plausible. (I haven’t verified your count parameter exactly.)

    However, I wonder why you’re trying to reinvent snapshots, a technology now built into several advanced filesystems, such as btrfs and ZFS?

    https://en.wikipedia.org/wiki/Btrfs#Subvolumes_and_snapshots

    btrfs is built into CentOS 7. While there have been some highly-publicized bugs in btrfs, they only affect the RAID-5/6 features. You don’t need that here, so you should be fine with btrfs.

    And if you really distrust btrfs, ZFS is easy enough to integrate into CentOS on-site.

    And if *that* is also out of the question, you have LVM2 snapshots:

    http://tldp.org/HOWTO/LVM-HOWTO/snapshots_backup.html

    Why reinvent the wheel?

  • Oh, I forgot to say, LVM2, ZFS, and btrfs snapshots don’t image the *entire* drive including slack space. They set a copy-on-write point which is near-instantaneous, so that whenever one of the current data blocks changes, its content gets copied to a new space on the disk and modified there, so that rolling back amounts to moving a bunch of pointers around, not downing the whole machine and wiping out your prior setup, including all that mail you’ve accumulated in the meantime.

    If you’re after some unstated goal, such as off-machine backups, there’s generally a way to send a copy of the snapshot to another machine, such as via SSH. This is also more efficient than copying a raw dd image. Not only does it skip over slack space, you can send the snapshot to another similar machine and “play back” the snapshot there, effectively mirroring the machine, taking only as much time as needed to transmit the *changes* since the last snapshot.

    If you’ve use a virtual machine manager with snapshotting features, these filesystems’ features are a lot like that. Quick, efficient, and quite robust.

  • This is CentOS7-armv7. Not all the tools are there. I keep getting surprises in some rpm not in the repo, but if I dig I will find it (but php-imap is NOT built yet and that I need).

    The base image is a dd, and you start with something like:

    xzcat CentOS-Userland-7-armv7hl-Minimal-1611-CubieTruck.img.xz | sudo dd of=/dev/sdb bs=4M; sync

    btw, this reports:

    0+354250 records in
    0+354250 records out
    3221225472 bytes (3.2 GB, 3.0 GiB) copied, 120.656 s, 26.7 MB/s

    Then you boot up (connected via the JART with a USB/TTL for a serial console).

    I want a drive image, and that is easy to do. I disconnect my drive from the CubieTruck, stick it into a USB/sata adapter, and I can image the whole drive.

    For just a development and snapshotting project, dd (and xzcat) do the job, and really what the Fedora-arm and CentOS-arm teams have been doing.

    I actually did this 2+ years creating Redsleeve6 images, but can’t find any of my notes. :(

  • I would recommend bsQ2 to keep the block sizes the same though not a huge diff just seems to be happier for some reason and add status=progress if you would like to monitor how it is doing. Seems the command you have should work otherwise.

  • So, given the fdisk output,

    Disk /dev/sdb: 111.8 GiB, 120034124288 bytes, 234441649 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x0000c89d

    Device Boot Start End Sectors Size Id Type
    /dev/sdb1 2048 1026047 1024000 500M 83 Linux
    /dev/sdb2 1026048 2074623 1048576 512M 82 Linux swap / Solaris
    /dev/sdb3 2074624 6268927 4194304 2G 83 Linux

    would

    countb68927

    ?

    oh, and this way, I can lay the image down on any drive. Even a mSD
    card (as that is the actual boot device, but the Cubie uboot (and linksprite) can run almost completely from a sata drive).

    thanks

  • thanks. I will look at this as I put the server into production. I
    never use all of the drive on my RSEL6 mailserver, so I would have space for snapshotting this one.

  • The dd blocksize has nothing to do with the disk sector size.

    the disk sector size is the number of bytes in a minimal read/write operation (because the physical drive can’t manipulate anything smaller).

    the dd blocksize is merely the number of bytes read/written in a single read/write operation. (or not bytes, but K, or Kb, or other depending on the options you use.)

    It makes sense for the bs option in dd to be a multiple of the actual disk block/sector size, but isn’t even required. if you did dd with a block size of, e.g., 27, it would still work, it’d just be stupidly slow.

    Fred

  • Kind of wondered about that.

    So the blocks reported by fdisk is what I should use as the count, as that matches the drive’s real block size?

    thanks

  • if you’re copying the entire device, you do not need to tell it how many blocks. just use a large-ish blocksize and let ‘er rip.

    for a single partition, you could use the blocksize and block number you get from fdisk. you would then need to say /dev/sda4, e.g., instead of /dev/sda.

    when copying an entire drive I tend to use 10M as the blocksize. using a large blocksize just reduces the number of read operations that are needed. that’s why a very small blocksize could slow down the copy, as it would require a whole lot more read operations.

  • Well, I only wanted to copy the used part of the drive which I try to keep small so I can still copy the image to an mSD card if I wish. So I
    have to supply the amount of the drive to copy. The bsQ2 went fast enough, but then I was only copying 3.2GB.

    thanks for the help.

  • personally, I would use ‘dump’ for ext? file systems, and xfsdump for XFS. this *just* backs up the inodes and directories, not raw blocks

  • Have you considered looking at Clonezilla? It does a nice job of imaging a disk and only saves the used part. It creates an image which can be restored and is bootable.

  • But I have a single image that I can use to build a working drive. Even on another drive, as long as it is more that 3.2GB. Also the partition table and uboot (but on Cubie’s uboot, you put the uboot as the ONLY
    thing on the mSD, and it switches over to the sata for all the partitions).

  • Yes, I looked at it a couple years ago, and I may well again as a disaster recovery tool. It takes more to set up than a dd of the drive
    (and xzcat compression for storage).

    I *DO* want a recovery strategy in place here.

    thanks

  • I have a Pogoplug (armv5) running Redsleeve 7 with a 2TB drive to back up to.

    AFTER I get my new mailserver working.

    Which probably won’t be until either someone builds php-imap for armv7, or I figure out how to get build or mock installed (both of which had failed dependencies) and do it myself.

    BTW, here is my Cubieboard2 ‘rack’. I should also put up my CubieTruck…

    http://medon.htt-consult.com/~rgm/cubieboard/cubietower-3.JPG

    Medon is the top one in the stack.