Using mvobj to rename large files slow and failed
I was trying to rename a 20GB file in a bucket using mvobj
but it takes a very long time and failed.
mvobj scratch:transfer/AMI_exp/exp_conv_real_RIRs.tar scratch:transfer/AMI_exp/exp_conv_real_RIRs_0.tar
Traceback (most recent call last):
File "/home/zhy/.local/bin/mvobj", line 47, in <module>
preserve_acl=True)
File "/home/zhy/.local/lib/python3.5/site-packages/boto/s3/bucket.py", line 873, in copy_key
query_args=query_args)
File "/home/zhy/.local/lib/python3.5/site-packages/boto/s3/connection.py", line 671, in make_request
retry_handler=retry_handler
File "/home/zhy/.local/lib/python3.5/site-packages/boto/connection.py", line 1071, in make_request
retry_handler=retry_handler)
File "/home/zhy/.local/lib/python3.5/site-packages/boto/connection.py", line 1030, in _mexe
raise ex
File "/home/zhy/.local/lib/python3.5/site-packages/boto/connection.py", line 944, in _mexe
response = connection.getresponse()
File "/usr/lib/python3.5/http/client.py", line 1242, in getresponse
response.begin()
File "/usr/lib/python3.5/http/client.py", line 311, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.5/http/client.py", line 272, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib/python3.5/socket.py", line 575, in readinto
return self._sock.recv_into(b)
File "/usr/lib/python3.5/ssl.py", line 929, in recv_into
return self.read(nbytes, buffer)
File "/usr/lib/python3.5/ssl.py", line 791, in read
return self._sslobj.read(len, buffer)
File "/usr/lib/python3.5/ssl.py", line 575, in read
v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out
And I opened the web interface at the same time to see a new file was first created and I thought the old file should be gone when it finishes. It feels like a cp A B && rm A
operation (and rm
failed) rather than a simple mv
. Is this normal at all?