Forum
Perch runway backup of assets stops working after a while
I've recently set up backup of my staging environment (for familiarisation and testing purposes), backup up to dropbox.
The cron job runs every 10 mins, with db backup every hour.
The db backup works fine.
The assets work fine for a while (few hours), then stops, with all subsequent asset backup tasks marked as "In Progress".
I have stopped, reset and restarted the backup process and witnessed the same behaviour.
There has been a couple of other similar posts about this but seemingly no resolution as yet (https://forum.grabaperch.com/forum/08-17-2016-perch-not-backing-up-assets). Please can you suggest what might be going wrong?
Perch Runway: 2.8.34, PHP: 5.5.33, MySQL: mysqlnd 5.0.11-dev - 20120503 - $Id: 15d5c781cfcad91193dceae1d2cdd127674ddb3e $, with PDO
Server OS: Linux, cgi-fcgi
Installed apps: content (2.8.34), assets (2.8.34), categories (2.8.34), perch_events (1.9.2), perch_gallery (2.8.6), refresher (2.1), perch_twitter (3.6.2)
App runtimes: <?php $apps_list = array( 'content','categories', 'perch_events', 'perch_twitter' ) ?>
PERCH_LOGINPATH: /perch
PERCH_PATH: /home/chrisgeo/public_html/tolmers-staging/perch
PERCH_CORE: /home/chrisgeo/public_html/tolmers-staging/perch/core
PERCH_RESFILEPATH: /home/chrisgeo/public_html/tolmers-staging/perch/resources
Image manipulation: GD
PHP limits: Max upload 256M, Max POST 256M, Memory: 256M, Total max file upload: 256M
F1: 6a33f95eca3667f9e0c39bf5ca2980fe
Resource folder writeable: Yes
DOCUMENT_ROOT: /home/chrisgeo/public_html/tolmers-staging
HTTP_HOST: www.tolmers-staging.chrisgeorge.org.uk
REQUEST_URI: /perch/core/settings/diagnostics/
SCRIPT_NAME: /perch/core/settings/diagnostics/index.php
Do you have any particularly large assets that might be causing a timeout?
The largest asset is about half a dozen pdfs at 4-5Mb. Is this large?
Are there any logs etc that I could find out what it is attempting to backup?
Have you checked your server's error log?
I only have limited access to the logs on my hosting... but there isn't anything obvious reporting there.
I've also just tried reducing the resource folder to 7 assets, all <60K, and reset the backup.
Again the db backup works fine, and the first asset backup works (backing up 6 files), but subsequent backups are marked "in progress" and that final file doesn't get backed up.
How frequent is your task scheduler running, and how often have you set the database to be backed up?
Task scheduler every 10 mins DB every 2 hours (was set to 1)
When you say:
What process did you follow to do this?
I moved all barr 7 assets out of the resource folder (into a new folder), then removed and recreated the backup plan.
However it's just occurred to me that the backup system is probably using records in the database to backup assets?
I was just trying to reduce the complexity of the problem whilst also wondering whether the backups would work with a smaller initial set, then gradually adding them.
Right, backup will run through the resources being managed by the system - it doesn't scan your file system. Moving files from under it could cause the backup to stall as it fails to find a file that should be available.
Right ok. I'll restore the assets and then check that all assets managed by the system exist in that folder.
Is there a way of enabling more logging from perch to show what it's trying to do (and potentially failing)?
There's not more logging that I'm aware of, no.
Hi Drew, Ok, I've sussed out what's going on with mine. There are a bunch of assets that are in the database but are not present in the resources folder (probably due to grabbing backups of the live db etc).
In PerchBackupPlan.class.php, the copy_resource function returns false if the file doesn't exist, but it also returns false if there was any other problem copying the file. The run_resource_backup function that calls copy_resource can't differentiate between file not exist 'false' and any other error 'false', therefore just keeps retrying the copy every time.
Also, in run_resource_backup, if copy_resource returns false, it doesn't do anything with it, ie marking the backup record as failed. However, I understand that this is probably done to force backup to retry next time.
However, this does mean that if the file doesn't exist (which I was only able to ascertain by adding debug statements into the php), it will retry the backup of that file or files forever more.
I'm happy to discuss this further offline with you :-)
Ok, I'll have it set the resource to AWOL of the file is missing.
Presumably no one is deleting files and then putting them back, so that sounds like it should be safe, and they always have the DB backup to reverse the process.
thats great, thanks Drew.
It would be really useful if on the backup page, hovering over each of thte backup events would list the asset files that were attempted to be backed up. I'm sure that would make diagnosing these sort of issues very much faster!
But thanks for the response and I look forward to getting the update.