Calculate optimum chunk size while copying single large file

Coding
1

I’ve a file of size 7 GB which I’ve to copy from local drive to server. I can’t use shutil.copy function because I want to show percentage for file copied by getting size of destination file at particular intervals. I’m doing it like this.

src = r"c:\test\file.mra"
dst = r"\\bkps\project\file.mra"
length = 1024 * 4

with open(src, 'rb') as fsrc:
    with open(dst, 'wb') as fdst:
        while True:
            buff = fsrc.read(length)
            if not buff:
                break
            fdst.write(buff)

Now this code is taking about 20-25 min to copy this large file, but when I copy this using normal windows copier It takes around 7 mins. I assume factor for this difference is the buffer size that I’m using (1024 * 4). How to decide or calculate the max buffer size to use for this job.

2

This link may be of some use to you:
http://www.onlamp.com/2005/11/17/tcp_tuning.html

You could also probably just play around a bit though and fairly quickly come up with a much faster transfer rate (by perhaps doubling your current buffer size for starters).

3

Thanks for the article link!