Htykuut

I am getting files from a host that will be consumed for another system. I realized I need to rename the files while it is being downloading from the remote server to avoid data corruption. Do you know if there is a way to achieve it, maybe the get() function in combination with StringIO ? or is not possible with fabric?

Edited: Please have a sample of my code.

# ~/fabfile.py

from fabric.api import task, env, run, settings, cd, put, get, execute



@task

def send_files():

'''

Send the downloaded files (found.txt) from remote to the server

'''

#  Get the sorted list of filenames (to send the files in order)

with settings(war_only=True):

    with cd(REMOTE_DIR):

        sorted_list = sort_files()

        for file in sorted_list:

            print(file)

            file = file.replace('n', '')

            #Something here to change the extension

            #when the download is complete change to the original extension

            # Example

            # get 427783.zip

            # change to 427783.crdownload

            # back to 427783.zip when is done.

           get(REMOTE_DIR + DESTINATION + '/' + file, INPUT_FOLDER + '\' + file)

I think I have to re-do my question: I would like to know when the download is completed using fabric ftp connection and the get() method, another service will pick up the file and would like to avoid the download is not completed yet.

Using wget will solve your problem. You can do the following:

 with connection(host=host, user=user) as c:

    save_path = '/var/www/new_name.extension'

    source_path = 'where_your_file_resides' 

    c.run('wget -p -O %s %s' %(save_path, source_path)

I did not find any suitable example to change the extension automatically when the downloading is still in process. So, I figured it out and download the file with another a temporary extension, once the get() function finish fabric will execute the next command, so I take advantage of this and rename the file.

Like this:

def rename_file(original, output):

    try:

        os.rename(original, output)

    except WindowsError:

        os.remove(output)

        os.rename(original, output)



@task

def send_files():

    with settings(war_only=True):

        with cd(REMOTE_DIR):

            sorted_list = sort_files()

            for file in sorted_list:

                old_file = REMOTE_DIR + DESTINATION + '/' + file

                temp_file = INPUT_OTDF_FOLDER + '\' + file + '.tmp'



                get(old_file, temp_file)

                rename_file(temp_file, new_file)