Create new rsync workflow for '--from-files' flag
In an attempt to support more diverse rsync environments, we're adding support for the '--from-files' flag which lets us specify input to rsync.
Note that this workflow comes with a slew of work that we need to do for it so there's additional work beyond just spawning rsyncs.
- Retrieve a file listing from the ingest server (tbd how this will look but most likely pre-split)
- Create a new class for transferring partial Bags
- The generic
BagTransfer
will run hashing and other operations which we don't want to do after each rsync - Needs to cleanup resources on success so we know what still needs to be transferred
- The generic
- Create a new class for Hashing a transferred Bag
- Should be pretty simple, hashing is normally handled by the Bucket
- Should also pass the computed hash to the ingest server
- Tie everything together using completable futures
- from retrieving the file listing to hashing, once everything is complete we want to continue with the rest of the flow