× Close

Please tell us about yourself so we can do our best for you.

This field is for validation purposes and should be left unchanged.

MozCon 2019: Luke Carthy – Killer CRO & UX Wins Using An SEO Crawler


Continuing on with our Ignite Search MozCon blog series, where we discuss some of the most influential and informative presentations at the 2019 MozCon Digital Marketing conference in Seattle, Washington. In this week’s blog post we will be discussing Luke Carthy’s presentation ‘Killer CRO & UX Wins Using An SEO Crawler’

Who is Luke Carthy

Before jumping into the presentation, we would first like to introduce Luke Carthy. Luke is a credible eCommerce consultant and international speaker that specialises in CRO and SEO.  With over 12 years of experience in this field, he has contributed and delivered powerful and quantifiable eCommerce sales growth for B2B and consumer brands such as CAT, Renault and Mayflex. He currently now an independent consultant for his own brand Luke Carthy.

luke carthy profile picture

Now For The Presentation Details

Luke begun his presentation with some figures on a company he previous was employed at – Mayflex. Mayflex is a leading distributor of converted IP solutions including infrastructure, networking and electronic security solutions located in Birmingham, United Kingdom. He revealed Mayflex had a 10% increase in traffic and a 250% increase in sales year on year.

He explained that it is important to know that traffic don’t always means sales, and sales don’t always mean traffic.

He continued to expressed his work ethic, describing himself as lazy and further spoke on behalf of many digital marketing specialist,

‘We are all trying to find ways to get us to the information that we need as quickly as possible.

This is why Luke loves custom extraction.

What is Custom Extraction?

Custom extractions are a powerful tool to pull specific information from webpages.  It allows you to scrape any data from the rendered HTML of webpages when crawling. It can be used by the following SEO crawling tools:

  • Deep Crawl
  • Screaming Frog
  • Sitebulb

Before diving into the implications of custom extraction, Luke brought us through his journey of analysing multinational electronic retail giant – Best Buy.

Best Buy has 236,000 indexed internal search URLS, with approximately 8 million visitors a month. Out of these 236,000 URLs, search results with zero results do not return 404s. Therefore there are thin result pages that are still receiving traffic.

So Luke walked us through his test. He crawled 4% of the 236,000 URLs and found 29 URLs returning ‘no results found.’ These URLs are indexed and meaning users are unable to find what they are looking for on the website. This represents missed opportunities whereby demand is outstripping supply.

How To Use Custom Extraction With Screaming Frog?

  1. Inspect the elements of the webpage, in this given example it will be the item count for a MacBook Air Latest from Best Buy:
    custom extraction screaming frog step 1
  2. Copy the HTML
    custom extraction screaming frog step 2
  3. Open up Screaming Frog and click the menu tab ‘Configuration’ and click ‘Custom’ then ‘Extraction’
    custom extraction screaming frog step 3
  4. Paste the HTML into the fields accordingly
    custom extraction screaming frog step 4

What Can You Do With Custom Extraction?

Custom extraction can help you understand your products and the customer journey. This includes your most popular or bestselling products or your worst selling products. However, it is important to observe the price level of these products, as your bestselling products may not necessarily suggest an increase in sales. This is relative to the number of products sold and the price label assigned to the products.

Furthermore, custom extraction does not only help you understand your products performance, but also your competitors. This is an excellent competition analysing tactic that can provide insights into your competitors’ strategies.

For example, given a shirt on a landing page from retail giants H&M, you will be able identify complementary data such as ‘what customers also purchased’ such as what shirt would go with what jeans.

In doing so, you will be able to set up and replicate a natural customer journey and boost conversions and sales. Through this competitive analysis you will also be able to see patterns and trends that similar to yours.

Luke explained through custom extraction you will be able to identify thin categories, traffic levels, how often search engine crawls them, the speed and the correlation between pages and categories.

He further stressed that this methodology is subjective to your objectives and recommended we look deeper into the vast applications available for custom extractions.

He suggested the following reads:

luke carthy recommended reads on custom extraction

So what SEO crawler tool should you use? Well Luke has also provided us the answer to that question too. Below you will find a matrix he had developed to comprehend the degree of use for the following SEO crawling tools:

  • Sitebulb
  • Screaming Frog
  • DeepCrawl
  • OnCrawl

luke carthy seo crawling tool matrix

What We Can Take Away From This Presentation?

As SEO and digital marketing specialists, we have assigned data as an imperative component to drive numerous activities that will yield tremendous results. Luke has provided us a quick and highly effective solution to find the right data. This solution is custom extraction and it allows us to further dive deeper into the performance of our website, the minds of customers and competitors’ strategies.

Recent Blog Posts