public final class ParallelPack extends Object

Our backup directories contain parallel directories with many hard links. rsync and tar both use extreme amounts of RAM to manipulate these directories and often fail or become extremely slow due to excessive swapping.

To work around this problem and be able to move directory trees from host to host, this tool will combine the set of directories and write them to System.out. This is similar to tar. The output is then unpacked using ParallelUnpack, which could be a direct pipe, through ssh, nc, or any other mechanism.

For efficiency, direct TCP communication is supported with the -h option.

It assumes that the file system is not changing, results of use on a changing filesystem is not defined.

AO Industries, Inc.
See Also:
  • Method Details

    • main

      public static void main(String[] args)
      Packs multiple directories in parallel (but not concurrently).
    • parallelPack

      public static void parallelPack(List<PosixFile> directories, OutputStream out, PrintStream verboseOutput, boolean compress) throws IOException
      Packs to the provided output stream. The stream is flushed and closed.