Multi-file Tarballs

Recently I made the tough decision to stop using Dropbox because I am paying for 2TB of space on iCloud through an Apple One subscription. During the process of copying files from Dropbox to iCloud, I encountered the iCloud file size limit[^1] of 50GB. This threw a wrench in my plans to migrate to iCloud.

Or did it?

Use case

You can combine the tar and split commands to spread a single tar archive across multiple files of a specified size. This can help you:

• Avoid maximum file size limits on a filesystem or cloud storage provider
• Split an archive into manageable "chunks" to be written/burned to physical media (more common in the past when burning backups to CD/DVD or tape backup)

How to create a multi-file tarball

You can run the split command on an existing tarball, while specifying the chunk size with the -b parameter (byte count). For example:

split -b 1024 my_archive.tar.gz my_archive.tar.gz.part_

This command would create multiple 1MB files out of my_archive.tar.gz, like this:

• my_archive.tar.gz.part_aa
• my_archive.tar.gz.part_ab
• my_archive.tar.gz.part_ac
• ...

You can create a tarball and split/chunk it with one like like this:

tar czvf my_archive.tar.gz /path/to/directory | split -b 20480M - my_archive.tar.gz.part_

This would create multiple files with a max size of 20GB (20480MB).

How to extract a multi-file tarball

Rejoin the parts into one large archive:

cat my_archive.tar.gz.part_* > my_archive.tar.gz

Join and extract the archive:

cat my_archive.tar.gz.part_* | tar xz

Final thoughts

I ultimately ended up switching back to Dropbox, mainly for performance reasons, but also because rclone does not yet (and may never) support iCloud Drive², but does support Dropbox rather handily³. However, I think this is still really good information to know.

What types of files can I store in iCloud Drive?, iCloud Drive FAQ ↩︎

Add support for Apple iCloud, GitHub Issues ↩︎

Dropbox, Rclone.org ↩︎