Forum
Image Uploads Corrupted
Hi,
We are trying to narrow down possible reasons why a number of images uploaded to a clients site appear to have been corrupted. The images in question have all ended up with the name dot_jpg.txt. Changing the extension to .jpg results in a horrible mess of the former image.
There have been over 900 images uploaded that do not have this problem so I am suspecting a different cause, however if there is anything in the diagnostics that could indicate what is going on that would be very helpful.
Perch: 2.8.34
Production mode: Production (100)
Installed apps: content (2.8.34), assets (2.8.34), categories (2.8.34), root_analytics (1.1.0), perch_blog (5.0), root_heartbeat (1.0.1)
DB driver: PDO
DB tables: perch2_blog_authors (2), perch2_blog_comments (0), perch2_blog_index (4998), perch2_blog_posts (156), perch2_blog_posts_to_tags (5), perch2_blog_sections (1), perch2_blog_tags (9), perch2_blogs (1), perch2_categories (7), perch2_category_counts (7), perch2_category_sets (1), perch2_content_index (723), perch2_content_items (63), perch2_content_regions (13), perch2_navigation (0), perch2_navigation_pages (0), perch2_page_templates (4), perch2_pages (4), perch2_resource_log (1255), perch2_resource_tags (3249), perch2_resources (4582), perch2_resources_to_tags (72462), perch2_root_analytics_cache (4), perch2_scheduled_tasks (30), perch2_settings (32), perch2_user_passwords (2), perch2_user_privileges (38), perch2_user_role_privileges (25), perch2_user_roles (2), perch2_users (2)
Users: 2
PHPMailer: 5.2.21
App runtimes:
<?php
$apps_list = array(
'content',
'categories',
'perch_blog',
'root_analytics'
);
Scheduled tasks for root_analytics: root_analytics_fetch_comparisons (1440 mins)
Scheduled tasks for root_heartbeat: root_heartbeat_broadcast_status (1440 mins)
Scheduled tasks for perch_blog: delete_spam_comments (1440 mins)
Editor plug-ins: redactor, markitup
H1: ac07d841c6c625b0980c5645f9f35f48
L1: 9806c53415b42db0c289fe14fc0a6986
F1: 6a33f95eca3667f9e0c39bf5ca2980fe
headerColour: #0B6271
content_singlePageEdit: 1
siteURL: /
hideBranding: 1
content_collapseList: 1
lang: en-gb
update_2.8.34: done
latest_version: 2.8.15
on_sale_version:
headerScheme: dark
root_analytics_namespace: Root Analytics
root_analytics_credentials_json: ###
root_analytics_time_period: 90
root_heartbeat_uid: b18b361f-37aa-5877-a5b8-9784a0e09939
root_heartbeat_signature: 24d4d2a2-ecd4-4e0c-b933-ad28c20418f3
root_heartbeat_master_server: https://2016.rootstaging.uk/perch/addons/apps/root_monitor/receiver.php
dashboard: 0
hide_pwd_reset: 0
content_hideNonEditableRegions: 0
content_frontend_edit: 0
logoPath: /admin/resources/coli-avatar3x.png
perch_blog_update: 5.0.1
perch_blog_post_url: /krokoblog/post.php?s={postSlug}
helpURL: mailto:design@rootstudio.co.uk
perch_blog_site_name:
perch_blog_slug_format: %Y-%m-%d-{postTitle}
perch_blog_akismet_key:
perch_blog_max_spam_days: 0
perch_blog_comment_notify: 0
root_analytics_ua: ###
root_analytics_view_id: ###
PERCH_DEVELOPMENT: 10
PERCH_STAGING: 50
PERCH_PRODUCTION: 100
PERCH_TZ: Europe/Moscow
PERCH_LOGINPATH: /admin
PERCH_PATH: /bhome/part3/03/vh48366/colicrocodile.ru/www/admin
PERCH_CORE: /bhome/part3/03/vh48366/colicrocodile.ru/www/admin/core
PERCH_RESFILEPATH: /bhome/part3/03/vh48366/colicrocodile.ru/www/admin/resources
PERCH_RESPATH: /admin/resources
PERCH_HTML5: 1
PERCH_DB_USERNAME: vh48366
PERCH_DB_SERVER: baze.zenon.net
PERCH_DB_PORT: 64000
PERCH_DB_DATABASE: vh48366
PERCH_DB_PREFIX: perch2_
PERCH_EMAIL_FROM: cms@colicrocodile.ru
PERCH_EMAIL_FROM_NAME: Коли Крокодил
PERCH_PRODUCTION_MODE: 100
PERCH_SCHEDULE_SECRET: dbJCywjwXGHqnmH4tdFbYyaY7EGtMfyA
PERCH_DEBUG:
PERCH_RUNWAY:
PERCH_ERROR_MODE: DIE
PERCH_DATE_LONG: %d %B %Y
PERCH_DATE_SHORT: %d %b %Y
PERCH_TIME_SHORT: %H:%M
PERCH_TIME_LONG: %H:%M:%S
PERCH_RUNWAY_ROUTED:
PERCH_STRONG_PASSWORDS:
PERCH_PREVIEW_ARG: preview
PERCH_TEMPLATE_PATH: /bhome/part3/03/vh48366/colicrocodile.ru/www/admin/templates
PERCH_DEFAULT_DOC: index.php
PERCH_DEFAULT_EXT: .php
PERCH_RWD:
PERCH_HTML_ENTITIES:
PERCH_SSL:
PERCH_STRIPSLASHES:
PERCH_PROGRESSIVE_FLUSH: 1
PERCH_PARANOID:
PERCH_FORCE_SECURE_COOKIES:
PERCH_PASSWORD_MIN_LENGTH: 6
PERCH_MAX_FAILED_LOGINS: 10
PERCH_AUTH_LOCKOUT_DURATION: 1 HOUR
PERCH_VERIFY_UPLOADS:
PERCH_AUTH_PLUGIN:
PERCH_DB_CHARSET: utf8
PERCH_DB_SOCKET:
PERCH_SESSION_TIMEOUT_MINS: 20
HOSTING SETTINGS
PHP: 7.0.15
Zend: 3.0.0
OS: FreeBSD
SAPI: apache2handler
Safe mode: not detected
MySQL client: mysqlnd 5.0.12-dev - 20150407 - $Id: b5c5906d452ec590732a93b051f3827e02749b83 $
MySQL server: 5.6.33-79.0-log
Free disk space: 349.85 GB
Extensions: Core, date, libxml, openssl, pcre, zlib, bcmath, calendar, ctype, curl, dba, dom, hash, fileinfo, filter, ftp, gd, gettext, SPL, iconv, imagick, session, intl, json, mbstring, mcrypt, standard, mysqlnd, mysqli, PDO, pdo_mysql, pdo_pgsql, pdo_sqlite, pgsql, Phar, posix, Reflection, imap, SimpleXML, soap, sockets, exif, timezonedb, tokenizer, xml, xmlreader, xmlrpc, xmlwriter, xsl, zip, apache2handler, memcache, Zend OPcache
GD: Yes
ImageMagick: Yes
PHP max upload size: 32M
PHP max form post size: 32M
PHP memory limit: 160M
Total max uploadable file size: 32M
Resource folder writeable: Yes
Session timeout: 24 minutes
Native JSON: Yes
Filter functions: Yes
Transliteration functions: Yes
UNIQUE_ID: WQGyDT5xViYAAS6SAmAAAAAA
HOME: /bhome/part3/03/vh48366
USER: vh48366
GECOS: colicrocodile.ru
SHELL: /usr/local/sbin/rtsh
LD_PRELOAD: /usr/local/lib/libpwd_dbenv.so
PERLLIB: /bhome/part3/03/vh48366/perllib
PROF_SRV: 1
HTTP_HOST: www.colicrocodile.ru
HTTP_X_ACLR_INTERNAL: /ZACLR
HTTP_CONNECTION: close
HTTP_COOKIE: PHPSESSID=1spih2v7hp99jjjni6laukpt66; cmsa=1; _ga=GA1.2.368066111.1485248911; _ym_uid=1485773801422822348
HTTP_UPGRADE_INSECURE_REQUESTS: 1
HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
HTTP_USER_AGENT: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/602.2.14 (KHTML, like Gecko) Version/10.0.1 Safari/602.2.14
HTTP_REFERER: https://www.colicrocodile.ru/admin/core/settings/diagnostics/
HTTP_ACCEPT_LANGUAGE: en-gb
HTTP_ACCEPT_ENCODING: gzip, deflate
HTTP_X_FORWARDED_FOR_ORIG: 81.130.131.129
PATH: /usr/bin:/bin
SERVER_SOFTWARE: Apache/2.2.23 (Zenon) PHP/7.0.15
SERVER_NAME: www.colicrocodile.ru
SERVER_ADDR: 62.113.86.38
SERVER_PORT: 80
REMOTE_ADDR: 81.130.131.129
DOCUMENT_ROOT: /bhome/part3/03/vh48366/colicrocodile.ru/www
SERVER_ADMIN: postmaster@colicrocodile.ru
SCRIPT_FILENAME: /bhome/part3/03/vh48366/colicrocodile.ru/www/admin/core/settings/diagnostics/index.php
GATEWAY_INTERFACE: CGI/1.1
SERVER_PROTOCOL: HTTP/1.0
REQUEST_METHOD: GET
QUERY_STRING: extended
REQUEST_URI: /admin/core/settings/diagnostics/?extended
SCRIPT_NAME: /admin/core/settings/diagnostics/index.php
PHP_SELF: /admin/core/settings/diagnostics/index.php
REQUEST_TIME_FLOAT: 1493283341.445
REQUEST_TIME: 1493283341
The only time we do anything remotely close to this is if a PHP file is uploaded as an image, at which point we add a
.txtextension.I wouldn't trust those files.
Ok, thanks for looking this over Drew. I didn't think it would be a Perch based issue - just needed to confirm before we ask the client to replace 100 images or so lol.
I'll close this now.
Hi Drew,
I think I have found the cause and have been able to replicate the issue on our own servers / locally. It looks like Russian character file names cause problems and result in the
dotjpg.txtfile name.To reproduce if you take a standard jpeg and give it a file name of
Капли от колик.jpgand upload using the Dropzone it appears to rename it. This seems to be fine until downloading the image and re-using it on a development install where the .txt extension must lose some image data and the corruption occurs (at least thats my best guess).Ok, I've tracked it down. The file name doesn't include any valid characters, so those get stripped, and you end up with just
.jpg.A file starting with a
.is a hidden file in unix and not the sort of thing that should be uploaded as an image, so Perch treats it as a hack attempt and changes the file to text.So the solution would be to make sure you have at least one latin character in the file name.
Thanks for looking into this Drew, that looks like the exact problem we are having. Is there any encoding settings that we could modify in PHP (as in php.ini or extensions rather than Perch Core) to avoid asking our Russian client to keep file names to Latin? It may be a little awkward to that explain their Russian language site on Russian hosting cannot handle Russian files.
If this isn't possible no worries.
What would you expect that file name to be converted to?
I am not really sure, obviously if assumptions start being made about odd characters the security measures put in place no longer do their job. How are url slugs generated from Russian characters or are they encoded using the same characters that are stripped from the file uploads?
If there is a way without breaking the security features, possibly a generic name like 'image-{x}' or 'unknown-{x}', or even put a single number in place so it becomes
1.jpg. Unless I have misunderstood the question.I'll see if we can transliterate the file names instead.
Awesome, thanks for looking into this :)