S3 Asset Preview URL not fully formed. I think I know why.
I'm uploading assets to an S3 bucket, which works great (thanks!), but the preview image in the Perch Editor is blank and shows this error:
Cannot open `s3: / / pec-static / sullivantiyika.jpg' (No such file or directory)
89KB
The bucket name is correct, and the file name is correct, but for some reason it's not building out the full S3 URL properly for the preview image.
It works fine otherwise- I'm using the output="filename" to build out an image URL with no problems. It's just that you can't see any of the images in the Perch editor because of that URL issue.
Any ideas how to work around this? Thanks!
Bucket Config
<?php
return [
'pec-static' => [
'type' => 'amazon_s3',
'web_path' => 'https://pec-static.s3.amazonaws.com',
'file_path' => 'pec-static',
],
];
Error Logs
It repeats this sequence for each asset:
[Fri Sep 18 12:43:44.889611 2015] [:error] [pid 24647] [client ::1:54217] PHP Warning: finfo_file(): Failed identify data 0:invalid mode 00 in /Library/WebServer/Documents/perch/core/lib/PerchImage.class.php on line 402, referer: https://localhost/perch/core/apps/content/edit/?id=23
[Fri Sep 18 12:43:45.191080 2015] [:error] [pid 24647] [client ::1:54217] PHP Warning: finfo::file(): Failed identify data 0:invalid mode 00 in /Library/WebServer/Documents/perch/core/lib/PerchUtil.class.php on line 1045, referer: https://localhost/perch/core/apps/content/edit/?id=23
[Fri Sep 18 12:43:45.464364 2015] [:error] [pid 24647] [client ::1:54217] PHP Warning: finfo_file(): Failed identify data 0:invalid mode 00 in /Library/WebServer/Documents/perch/core/lib/PerchUtil.class.php on line 1055, referer: https://localhost/perch/core/apps/content/edit/?id=23
[Fri Sep 18 12:43:46.121682 2015] [:error] [pid 24647] [client ::1:54217] PHP Warning: mime_content_type(): Failed identify data 0:invalid mode 00 in /Library/WebServer/Documents/perch/core/lib/PerchUtil.class.php on line 1078, referer: https://localhost/perch/core/apps/content/edit/?id=23
Diagnostics
Perch Runway: 2.8.13, PHP: 5.5.27, MySQL: mysqlnd 5.0.11-dev - 20120503 - $Id: 15d5c781cfcad91193dceae1d2cdd127674ddb3e $, with PDO
Server OS: Darwin, apache2handler
Installed apps: content (2.8.13), assets (2.8.13), categories (2.8.13)
App runtimes: <?php $apps_list = array( 'content', 'categories', );
PERCH_LOGINPATH: /perch
PERCH_PATH: /Library/WebServer/Documents/perch
PERCH_CORE: /Library/WebServer/Documents/perch/core
PERCH_RESFILEPATH: /Library/WebServer/Documents/perch/resources
Image manipulation: GD
PHP limits: Max upload 12M, Max POST 8M, Memory: 128M, Total max file upload: 8M
Resource folder writeable: Yes
HTTP_HOST: localhost
DOCUMENT_ROOT: /Library/WebServer/Documents
REQUEST_URI: /perch/core/settings/diagnostics/
SCRIPT_NAME: /perch/core/settings/diagnostics/index.php
Is
pec-staticyour bucket name?What would a fully formed URL to one of your images look like?
Thanks Drew- pec-static is the correct bucket name. Images get uploaded in their properly.
If I remove the output="filename":
I get this (which is correct):
I think this may be the difference between serving files via S3 vs CloudFront.
For CF you'd have a domain like
then the full image path would be
Ah ha - try setting:
I was really hoping that would work, Drew, but I got the exact same behaviour. I tried experimenting with some different settings based on that tweak and didn't get anything that worked.
I do think there is something funky going on with the MimeType, as indicated in the Log files. If you watch the video below of me using the interface, you'll see the first image fails. I save it as another image, it still fails, then I save it with a different Jpeg compression setting and it works OK for the thumbnail preview. Until I save the region, at which point the thumbnail fails again. Perhaps this will give you some insight as to what's going on.
Screengrab of thumbnail issue
What path do you get now?
If I change the setting as you suggested to:
I get this as the file path (incorrect):
Just for clarity's sake, with the config like so:
I get a path that looks like this (correct):
I think there are two problems going on-
1) Either ImageMagick or something else are having problems trying to parse Jpeg files with the wrong MimeType or something like that.
2) Some images work fine (I guess the ones without the aforementioned Mime problem)- the preview thumbnail is visible and the URL is correct- until I click "Save" or "Save and Add another". Something about that particular operation causes the link for any thumbnail that were working to break.
I opened up my S3 bucket to full permissions for everyone just to make sure that wasn't the problem and it didn't seem to affect anything.
Ah, ok, so it's only some images and not others?
I did some more research and there are two issues at play here. Everything works fine using local 'Default' buckets, and breaks with remote S3 buckets.
Issue #1 What's happening (from viewing the Perch source) is before the Save, Perch is referencing a local thumbnail URL (which works). Once it is saved, it changes to reference the remote S3 thumbnail, which has a malformed URL (as we talked about near the beginning of this thread). I think this is problem #1 going on. The URL for the thumbnail is being generated incorrectly somewhere along the way when using S3 buckets. If I was more comfortable with PHP I could probably sort through this and see where it's going wrong, but I suck at PHP so I'm just grasping at straws.
If I can track down where the remote S3 thumbnail URL is being generated, I'll let you know. That's the key to fixing this first problem.
Issue #2 On remote S3 buckets only, some images are not getting written to S3 properly. I'm not sure why this is, but you can see clearly what I'm talking about below. In the first screenshot, I've saved an Image to Perch using a local default bucket. Check out the image dimensions properly displayed:
If I switch over to a remote S3 bucket and upload the exact same file, I get this:
I used to write a Photoshop API called DaisyAPI and I remember having a strange issue with MimeTypes. I can't remember what was going on, but somehow when certain files are being written to S3, the MimeType is being mis-identified, causing the image upload to fail (like thinking it's a PNG when in fact it is a jpg).
If you want to see what I'm talking about, here is a file that will cause the remote S3 upload to fail: https://fm001-forumimages.s3.amazonaws.com/Ashe_Stacey.jpg
If you want access to my S3 bucket for testing with, I'm happy to give you that info privately so you don't have to create your own AWS account + bucket, etc.
That's a very big image - how much memory do you have allocated to PHP? It could be maxing out and failing to resize the image.
Unfortunately it does it on any filesize:
Here is a look at the code from the Perch Content Editor. Images that invoke the Mime error don't even get a src URL written to them. Images without the Mime error get the Src URL written properly:
As a reminder, the only difference between the two images above is one was 're-saved' with a different JPEG compression which seems to correct the Mime problem.
What is the source of these bad images?
I don't know. I ran them both through online Mime checker (https://mime.ritey.com/) and they both present as image/jpeg. I'm going to tell the client to re-save their images if they have a problem with the thumbnail preview.
I'm guessing you could code around this very peculiar edge case if you had the time, but I'd leave it until other folks are complaining and mark it resolved for now.
Many, many thanks for suffering through this with me.
It's not something we've heard before, but we'll keep an eye out.
I've run into the exact same issue here.
Sure enough, when JPGs are saved with Photoshop's standard File > Save As dialog they fail when uploading to Perch/S3 exactly as indicated above.
When I saved those same JPGs with File > Export > Save for Web, the upload completes successfully.
Sweet! Now I'm not the only one that has had this problem. I still haven't been able to figure out why some JPEGs cause problems on S3. The fact that it never happens locally and sometimes on S3 (network) makes me feel like it's an asynchronous issue. I thought PHP was synchronous, but perhaps they're using some sort of async library.
Hey Perch Humans- Client is complaining about this issue. If you could point out where in your source code the preview URLS are formed, I'm going to hire a PHP developer to try and sort this out. I will share the results.
Thanks!
Surely it's an issue with the files, not the URLs?
We can't support you in modifying Perch core, or once you have done so.
I would agree that yes, it's a problem with the files. But I can not control what gets put into the system. These are unskilled users who are uploading employee photographs into a company website and there is no telling where the images come from. They wouldn't know how to use Photoshop and probably have no Image Editors on their computers.
I'm imagining a try/catch/fail routine that will use ImageMagick to re-convert the original image if a preview URL appears to be malformed.
I don't expect any support on modifying the core, but would love for you guys to take this problem seriously- that would be much easier for me! :)
The thing that makes me think it may not be completely an issue with the files is that it works fine on local storage but fails through S3- I should have mentioned that one caveat.