Mount and sync a USB drive on RPi Ubuntu with GPIO

Actions:image
  1. So first the USB is plugged in.
  2. It is automounted using a GPIO button press. Flash red if this fails.
  3. Blue light will flash light during rsync cp (copy). If an error occurs show red LED and turn off blue LED.
  4. Send email with results on completion.
  5. Unmount USB and show steady blue LED indication turn off blue LED.

Ok lets do it
  • For auto mounting, the package usbmount seems too problematic (won’t mount NTFS drives) after trying these instructions.
  • it took much longer than I’d have liked to get to the point of using this mount command with sudo:
    mount -L SneakerNet -o uid=ubuntu,gid=deluge,dmask=007,fmask=117 /media/USB

This allows me to mount the USB by it’s label (SneakerNet) alone. This I think is ideal if I want to use a different USB drive.

  • I can put the mount command in a script and chmod 750
    #!/bin/bash
    # mount USB drive
    
    mount -L SneakerNet -o uid=ubuntu,gid=deluge,dmask=007,fmask=117 /media/sneaker$
  • The next problem is auto mounting a USB drive. In ubuntu server this is surprisingly difficult, this page shows how incomplete this is. udev while promising doesn’t work for NTFS (this came close), which took me a long time to realise – even invoking a python script from a bash script called by udev… pmount requires the device name, which can change depending on whats plugged in, usbmount doesn’t work, udisks2 is out (doesn’t allow the required options for mounting) :/
  • this looks interesting
  • for exfat, this could work
    sudo apt-get install exfat-utils exfat-fuse
  • So I’m trying a python implementation of udev (sigh x2). Trying this example.
    • Install pyudev
      sudo pip install pyudev
  • I think I’ll skip auto mounting and try mounting from a button press (in python as previously installed).
    #!/usr/bin/env python
    
    # mount USB drive
    
    import subprocess
    
    import time
    
    mountCommand = "mount -L SneakerNet -o uid=ubuntu,gid=deluge,dmask=002,fmask=113 /media/SneakerNet"
    
    time.sleep(1)
    
    process = subprocess.Popen(mountCommand.split(), stdout=subprocess.PIPE)
    
    output = process.communicate()[0]

    Well that little script took all of 5 minutes and negated 2 days work..

  • Now I’ll add in rsync, logging and email.
    #!/usr/bin/env python
    import subprocess
    import os
    from email.mime.text import MIMEText
    
    mountCommand = ["mount", "-L", "SneakerNet", "-o", "uid=ubuntu,gid=deluge,dmask=002,fmask=113", "/media/SneakerNet"]
    umountCommand = ["umount", "/media/SneakerNet"]
    mkdirCommand = ["mkdir", "/media/SneakerNet/Done"]
    
    if os.path.ismount("/media/SneakerNet/") == 0:
        MountProcess = subprocess.Popen(mountCommand, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        MountData, MountError  = MountProcess.communicate()
        if len(MountData) == 37:
            mountfail = 1
            logs = "No USB with label SneakerNet found."
        elif len(MountData) == 0:
            logs = "Mounted"
            mountfail = 0
        else:
            logs = "Coding error or no su maybe?)"
            mountfail = 1
    else:
        logs = "Already mounted"
        mountfail = -1
    
    rsyncCommand = ["rsync", "--delete", "-ahP", "--no-D", "--log-file=/media/SneakerNet/RsyncLog.txt", "/media/Data/Deluge/Done/", "/media/SneakerNet/Done"] #--dry-run #still have -P
    
    if mountfail < 1:
        if os.path.isdir("/media/SneakerNet/Done") == 0:
            subprocess.Popen(mkdirCommand)
            logs += "\nMade Directory /media/SneakerNet/Done"
        if os.path.isfile("/media/SneakerNet/RsyncLog.txt") == 1:
            with open("/media/SneakerNet/RsyncLogArch.txt", "a") as RsyncLogArch, open('/media/SneakerNet/RsyncLog.txt', 'r') as rsynclog:
                for line in rsynclog:
                    RsyncLogArch.write(line)
                logs += "\nMoved old rsync log to archive"
        with open('/media/SneakerNet/RsyncLog.txt', 'w') as rsynclog:
            rsynclog.write("Python output:\n")
            logs += "\nEmptied Log for rsync:\n"
            rsynclog.write(logs)
        print logs #for GUI...
        subprocess.call(rsyncCommand) ##### USING CALL (only for GUI)
    
        rsynclog = open('/media/SneakerNet/RsyncLog.txt', 'rb')
        msg = MIMEText(rsynclog.read())
        rsynclog.close()
        msg["From"] = "python@parko.xyz"
        msg["To"] = "phaseform@gmail.com"
        msg["Subject"] = "RPi rsync log"
        emailProcess = subprocess.Popen(["/usr/sbin/sendmail", "-t", "-oi"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
        emailOutput = emailProcess.communicate(msg.as_string())
    
        umountProcess = subprocess.Popen(umountCommand)
        mountData, mountError = umountProcess.communicate()
        print "Done..." #for GUI...
        
    else:
        msg = MIMEText(logs)
        msg["From"] = "python@parko.xyz"
        msg["To"] = "phaseform@gmail.com"
        msg["Subject"] = "rsync did not run"
        emailProcess = subprocess.Popen(["/usr/sbin/sendmail", "-t", "-oi"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
        emailOutput = emailProcess.communicate(msg.as_string())
        
    """
    -moove over script to lights
    """

    Currently I have a problem with rsync where it won’t stop when the drive is full..? this is compounded by my POS Telstra cable modem – which the RPi is connected to – which hangs on my SSH connection all the time… I’ll be swapping it out for an old Motorola Surfboard modem soon. Despite this rsync bug I’m testing with:

    sudo python "/home/ubuntu/gpio/DelugeRsyncEmailGUI.py"

    as from here and here.

    I added an rsync command before any writing is done to the USB, to remove any partial files, in case the USB is totally full, as here.
    Still need to add in flashing for the LED and other better indications of status.

  • Using cp instead of rsync due to an rsync bug where it doesn’t stop syncing when the target is at capacity. Now my problem is that there are a bunch of files with size = 0 which occur when the USB device reaches capacity. Although I think that may be the only problem with this method.
    #!/usr/bin/env python
    import time
    import RPi.GPIO as GPIO
    import subprocess
    import os
    from email.mime.text import MIMEText
    import datetime
    GPIO.setmode(GPIO.BOARD)
    
    GPIO.setup(12, GPIO.OUT) #Red LED for fauilure
    GPIO.setup(13, GPIO.IN, pull_up_down=GPIO.PUD_DOWN) #rsync button
    GPIO.setup(16, GPIO.OUT) #rsync blue LED
    GPIO.setup(22, GPIO.IN, pull_up_down=GPIO.PUD_DOWN) #used to keep CPU usage low
    
    mountCommand = ["mount", "-L", "SneakerNet", "-o", "uid=ubuntu,gid=deluge,dmask=002,fmask=113", "/media/SneakerNet"]
    #mount -L SneakerNet -o uid=ubuntu,gid=deluge,dmask=002,fmask=113 /media/SneakerNet
    umountCommand = ["umount", "/media/SneakerNet"]
    mkdirCommand = ["mkdir", "/media/SneakerNet/Done"]
    rsyncCLEANCommand = ["rsync", "--delete", "-rptgo", "--existing", "--ignore-existing", "/media/Data/Deluge/Done/", "/media/SneakerNet/Done"]
    #rsync -rptgovh --delete --existing --ignore-existing /media/Data/Deluge/Done/ /media/SneakerNet/Done
    cpfilesCommand = ["cp", "-nvpR", "/media/Data/Deluge/Done", "/media/SneakerNet"]
    #cp -nvpR /media/Data/Deluge/Done/ /media/SneakerNet/Done
    
    def rsync_callback(channel): #rsync sequence
        GPIO.output(16, 0) #blue LED on.
        GPIO.output(12, 1) #Red off.
        if os.path.ismount("/media/SneakerNet/") == 0:
            MountProcess = subprocess.Popen(mountCommand, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
            MountData, MountError  = MountProcess.communicate()
            if len(MountData) == 37:
                mountfail = 1
                logs = "No USB with label SneakerNet found."
                GPIO.output(12, 0) #Red on
            elif len(MountData) == 0:
                logs = "Mounted"
                mountfail = 0
            else:
                logs = "Coding error or no su maybe?)"
                mountfail = 1
                GPIO.output(12, 0) #Red on
        else:    
            logs = "Already mounted"
            mountfail = -1
        
        if mountfail < 1:
            if os.path.isdir("/media/SneakerNet/Done") == 0:
                subprocess.Popen(mkdirCommand)
                logs += "\nMade Directory /media/SneakerNet/Done"
            rsynccleanProcess = subprocess.Popen(rsyncCLEANCommand, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) #clean Done Dir in case disk is totally full..
            rsyncData, rsyncError = rsynccleanProcess.communicate()
            today = datetime.datetime.now()
            with open('/media/Data/Deluge/RsyncTempLog.txt', 'w') as rsynclog:
                rsynclog.write("\nDate:")
                rsynclog.write(str(today))
                rsynclog.write("\nLogs:\n")
                rsynclog.write(str(logs))
                rsynclog.write("\nNow for cp:\n")    
            cpProcess = subprocess.Popen(cpfilesCommand, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) 
            cpData, cpError = cpProcess.communicate()
            #rsynccleanProcess = subprocess.Popen(rsyncCLEANCommand, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) #remove partials
            #rsyncData, rsyncError = rsynccleanProcess.communicate()
            with open('/media/Data/Deluge/RsyncTempLog.txt', 'a') as rsynclog:
                rsynclog.write(str(cpData))
                rsynclog.write("\ncp done ^.\nsend email and finish (hopefully...)\n")
            rsynclog = open('/media/Data/Deluge/RsyncTempLog.txt', 'rb')
            msg = MIMEText(rsynclog.read())
            rsynclog.close()
            msg["From"] = "python@parko.xyz"
            msg["To"] = "phaseform@gmail.com"
            msg["Subject"] = "RPi sync log"
            emailProcess = subprocess.Popen(["/usr/sbin/sendmail", "-t", "-oi"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
            emailOutput = emailProcess.communicate(msg.as_string())        
            umountProcess = subprocess.Popen(umountCommand)
            mountData, mountError = umountProcess.communicate()
            GPIO.output(16, 1) #blue LED off.
        else:
            msg = MIMEText(logs)
            msg["From"] = "python@parko.xyz"
            msg["To"] = "phaseform@gmail.com"
            msg["Subject"] = "rsync did not run"
            emailProcess = subprocess.Popen(["/usr/sbin/sendmail", "-t", "-oi"], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
            emailOutput = emailProcess.communicate(msg.as_string())
            
        
    GPIO.add_event_detect(13, GPIO.RISING, callback=rsync_callback, bouncetime=300)
    
    try:
        GPIO.output(12, 1)
        GPIO.output(16, 1) #blue LED off.
        GPIO.wait_for_edge(22, GPIO.FALLING) #never going to happen
    
    except KeyboardInterrupt:
        GPIO.remove_event_detect(13)
        GPIO.remove_event_detect(22)
        GPIO.output(16, 1) #blue LED off.
        GPIO.cleanup() # clean up GPIO on CTRL+C exit
        
    GPIO.output(16, 1) #blue LED off.
    GPIO.remove_event_detect(22)
    GPIO.remove_event_detect(13)
    GPIO.cleanup() # clean up GPIO on normal exit

1 thought on “Mount and sync a USB drive on RPi Ubuntu with GPIO”

Leave a Reply

Your email address will not be published. Required fields are marked *