Asymmetric Encryption For Very Large Tar File

Home » CentOS » Asymmetric Encryption For Very Large Tar File
CentOS 14 Comments

Hello CentOS list, I have a requirement that I need to use encryption technology to encrypt very large tar file on a daily basis. The tar file is over 250G size and those are data backup. Every night the server generated a 250G data backup and it

14 thoughts on - Asymmetric Encryption For Very Large Tar File

  • It seems likely that openssl hasn’t been compiled with large-file support. Not so uncommon with RH5.

    Can you send the output to stdout and redirect? Or if that fails then send to stdout and filter via “dd” to write to the file. Now at this point openssl is only writing to a pipe and won’t hit the 2G limit.

  • What happens if you use a pipeline or redirection instead of the -in and -out files? I regularly write large tapes with something like:
    openssl aes-256-cbc -salt -k password lesmikesell@gmail.com

  • Am 17.12.2014 um 18:42 schrieb Les Mikesell :

    Furthermore – is there the need to use “one” big tar file? Despite having a capable workstation/server handling such big files, it has also advantages splitting such backups (e.g. man split) …

  • Is it possible for you to use gpg? You could do something like:
    tar zcf /something – | gpg -e -r otherkey | cat – > backup.tgz

    Regards

  • I would rather work on single files or tars on directory basis. Using a single big file creates a very “large” single point of failure. Or use an encrypted file system (of course, also a single point of failure, but probably better handling).

    Kai

  • Hello,

    The bad points with using an encrypted fs maybe in the OT case, is that to move the encrypted file to somewhere else, you need to move the hardware containing the fs :-(. Also, it doesn’t allow changing the encryption key very often. I think an encrypted fs addresses other security/confidentiality issues, but then the OT should be more precise about his needs/the context.

    My 2 cents.

    Regards,

  • Which might be as simple as swapping a USB key or portable drive.

    Yes, how the backup copies will be managed after encryption would have a lot to do with picking the most convenient approach. One thing that would be possible on an encrypted file system would be using a backup approach that stores multiple copies, de-dupinng unchanged files as you can do with rsync, rdiff-backup, BackupPC, etc. Those can only work if the software involved sees the unencrypted files.

  • From: Kai Schaetzl

    Afio is supposed to have better error handling than tar, and other nice options like pgp.

    JD

  • GPG is really what you want to be using for this. OpenSSL is a general toolkit that provide a lot of good functions, but you need to cobble some things together yourself. GPG is meant to handle all of the other parts of dealing with files.

    I will expand on what someone else mentioned — asymmetric encryption is not meant for, and has very poor performance for encrypting data, and also has a lot of limitations. The correct way to handle this is to create a symmetric key and use that to encrypt the data, then use asymmetric encryption to encrypt only the symmetric key.

    GPG takes care of this all internally, so that’s what you should be using.

    ❧ Brian Mathis
    @orev

  • that article is only talking about openssl… openssh, gpg, and others use their own crypto implementations.

    not CentOS/rhel specific, but.. Intel claims OpenSSL v1.0 has direct support, 0.9.8k+ has support via a patch.