What Does "Loaded In A Browser" Mean On The Crawl Screen?
Why we fetch some pages twice, what it means when scripts add product cards or text after a page loads, and why that text does not count toward your score.
Short Answer
We crawl every page as plain HTML, the way a search engine first gets it. We also load a sample of your pages in a real browser to see what your theme's scripts add afterwards. If scripts add product cards, text or a heading, the crawl screen says so, because that part of the page is less certain to be read. Only the plain HTML counts toward the score.
When a browser asks for a page, the store sends HTML first. Then the page's scripts run, and they can add, move or hide things. What a shopper sees is both together. A search engine gets the first HTML at once, and may or may not run the scripts later.
What We Do
Every page is crawled as plain HTML first. Pages that look alike share a layout, and for each layout we load a few pages in a real browser as a sample. A page whose layout no other page shares is always loaded. If a sample shows scripts adding something a shopper sees, the other pages of that layout are loaded too.
When the crawl finishes, one line under the counts sums it up. It reads one of these ways, with your own numbers:
- "We loaded ... pages in a browser and found nothing that scripts add."
- "We loaded ... pages in a browser. On ... of them, scripts add product cards, text or a heading after the page loads."
- "We could not load ... pages in a browser, so we have not checked what scripts add to them."
Why Script Text Does Not Count
Your score counts what is in the first HTML. Text that only appears after a script runs reaches some search engines later and others not at all, so we do not count on it. The browser load is there to tell you when this applies to you, not to change the score.
If scripts on your store add the product cards themselves, the page may arrive with almost nothing on it. That is worth raising with whoever looks after your theme.
The Same Idea On Competing Pages
The pages you are compared with are crawled the same way. On How Your Page Compares, a page read after its scripts ran is marked Crawled After Running Scripts. A page whose product grid only appears after scripts, and which we could not load that day, is marked Could Not Crawl: Grid Needs Rendering and left out of the averages.
A Daily Limit Applies
Loading a page in a browser costs more than fetching its HTML, so each account has a daily number of them, set by plan. If the day's number runs out, pages are crawled as plain HTML and marked as not checked in a browser. They are never marked as clean.
Still Stuck?
When a screen in the app shows Tell Us What Went Wrong, use it: the note it opens carries the page you were on, so we can look at the same thing. Otherwise, write to us at help@collectionoptimizer.com, and say which screen you were on and what it said.