Forum

Thread tagged as: Runway

Need help rethinking Runway collection "related items" code

One of my sites is experiencing much higher traffic than normal today (1,000+ concurrent users -- "normal" is 5). When the spike started, load time for the page in demand went from 1s to 40s until I removed a bit of code from my Runway page template, at which point load time dropped right back down to 1s.

The bit of code I removed searches through all collection items (currently 1,200 items) to find other items with the same categories as the item being viewed. Basically I'm just putting all of the categories for the current item into an array to be used with the category option for perch_collection() which retrieves the related posts. Here's the page template:

$result = perch_collection('Blog Posts',array(
  'template' => 'blog/posts/_post.html',
  'filter'=>'postSlug',
  'match'=>'eq',
  'value'=>perch_get('post'),
  'skip-template' => true,
  'return-html' => true,
  'count' => 1,
));

perch_layout('global/header',[
  'blog-post' => true
]);

echo $result['html'];

// related posts
$cats = perch_collection('Blog Posts',array(
  'template' => 'blog/_get_cat_array.html',
  'filter'=>'postSlug',
  'match'=>'eq',
  'value'=>perch_get('post')
),true);
$cats = explode(",",$cats);
perch_collection('Blog Posts',array(
  'template' => 'blog/posts/_related_posts.html',
  'count'=>3,
  'sort'=>'postDateTime',
  'sort-order'=>'RAND',
  'category'=>$cats,
  'filter'=>[
    [
      'filter'=>'postSlug',
      'match'=>'neq',
      'value'=>perch_get('post')
    ],
    [
      'filter'=>'postDateTime',
      'match'=>'lte',
      'value'=>date('Y-m-d H:i:s')
    ]
  ],
));

perch_layout('global/footer',[
  'scripts'=>'<script type="text/javascript" src="/js/rrssb.min.js?v=2018021201"></script>'
]);

...

This is the part of that code that was causing high load:

// related posts
$cats = perch_collection('Blog Posts',array(
  'template' => 'blog/_get_cat_array.html',
  'filter'=>'postSlug',
  'match'=>'eq',
  'value'=>perch_get('post')
),true);
$cats = explode(",",$cats);
perch_collection('Blog Posts',array(
  'template' => 'blog/posts/_related_posts.html',
  'count'=>3,
  'sort'=>'postDateTime',
  'sort-order'=>'RAND',
  'category'=>$cats,
  'filter'=>[
    [
      'filter'=>'postSlug',
      'match'=>'neq',
      'value'=>perch_get('post')
    ],
    [
      'filter'=>'postDateTime',
      'match'=>'lte',
      'value'=>date('Y-m-d H:i:s')
    ]
  ],
));

...

This is _get_cat_array.html:

<perch:categories id="categories" set="blog" >blog/<perch:category id="catSlug" type="slug" /><perch:if not-exists="perch_item_last">,</perch:if></perch:categories>

...

Is there some better way of doing this which won't cause so many round trips to the db? Thanks for the help!

Shane Lenzen

Shane Lenzen 18 points

  • 3 months ago
Drew McLellan

Drew McLellan 2578 points
Perch Support

I don't see many roundtrips to the db - just a very complex query.

How often does the result change? Have you thought about just caching it?

Montgomery Lewis

Montgomery Lewis 2 points
Registered Developer

Shane, I just had this same issue and ended up putting the time consuming code onto a support page that wrote the resulting html to a .txt file. Then on the page-in-question, read in the .txt file. I have a cron job set to run hourly to load the support page to keep the information up to date. Really speedy now. The data is not updated all that frequently so I'm still thinking of ways to improve the writing of the text file part. Ideally, the text file would be written any time the data is changed. Know what I mean?

Thanks Drew and Montgomery. Both great ideas, caching definitely sounds like a potential solution. I'm not sure Montgomery's solution will work in my case because we have 1,200 articles and each one will need its own related articles. The idea behind the original code was to display related articles at random to improve pageviews to all articles. So, it sounds like I need to:

  1. Cache related posts for each individual article
  2. When an article is viewed, check the last time the related posts were cached
  3. If the cache is older than an hour, refresh the cache

If anyone has done something similar and can point me in the right direction, I would greatly appreciate it!