In 2019, Microsoft Bing announced new Bingbot user-agent names that fit better with its evergreen Bingbot crawling and rendering service. Microsoft’s Fabrice Canel has now said that by the Fall of 2022 the old user-agent will stopped being used and the search company will transition to the new user-agents fully.
Old user-agent. Microsoft said it will stop using its historical user-agent by Fall 2022. That user-agent looks like this: Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
New user-agent. Bing will use a user-agent that identifies the specific version of Microsoft Edge is crawling your site. Here is the format for both desktop and mobile:
Mobile – Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
“W.X.Y.Z” will be substituted with the latest Microsoft Edge version Bing is using, for eg. “100.0.4896.127″.
Do we need to worry? Most sites do not need to worry. Microsoft said “For most web sites, there is nothing to worry as we will carefully test the sites to dynamically render fine before switching them to Microsoft Edge and our new user-agent.” But if you have hardcoded any user agents into your scripts, you will need to revise those scripts to ensure BingBot can continue to crawl your site.
How to test. Bing previously said you can test it by installing the new Microsoft Edge browser “to check if your site looks fine with it.” Bing said “if it does then you will not be affected by the change.” “You can also register your site on Bing Webmaster Tools to get insights about your site, to be notified if we detect issues and to investigate your site using our upcoming tools based on our new rendering engine,” Bing added.
Bing added “we will carefully test websites before switching them to our new user-agent Bing Webmaster Tools URL Inspection has already started using the new desktop user-agent for the Live URL Test to help you investigate potential issues.”
Google also. Google is also migrating to the new GoogleBot this month. Google is currently testing the new user agents, so you may be able to see them in your log files. I do not believe it is fully rolled out yet for Google.
Why we care. You probably should have been prepared for this change, since it was announced back in 2019. But in any event, this change can impact your site if you had any user agent detection methods for BingBot. Make sure to test your site to see if it supports the new user-agent. Most sites probably do not need to worry about this but you have done any advanced bot detection, you may need to take steps to update those scripts.
Messy SEO is a column covering the nitty-gritty, unpolished tasks involved in the auditing, planning, and optimization of websites, using MarTech’s new domain as a case study.
This installment of “Messy SEO” details my process of working with our team to analyze indexing patterns for MarTech’s pages. In Part 6, we discussed the necessity of creating pillar pages to establish a better site hierarchy and rank for our most relevant topics.
MarTech.org has had many indexing issues since its creation last year. The most pressing one lately is that Google seems to be prioritizing outdated content in the SERPs, meaning many of the (now redirected) Marketing Land and MarTech Today URLs are still populating the index. As a result, the majority of MarTech’s top-performing pages are irrelevant to our brand as it exists today.
One of the ways we’re addressing this issue is by creating pillar pages that center on the main industry topics we cover at MarTech. This will help us establish a hierarchy of relevant topics.
We’ve primarily focused on Google’s indexation throughout this process, neglecting to review the ways other search engines have treated our content. So, we decided to compare the MarTech, Marketing Land, and MarTech Today data from Google with that from Microsoft Bing – and the discrepancies were telling.
Indexing status almost a year after migration and consolidation
There have been a lot of changes to MarTech’s indexing since the migration, most notably the title change issues. Thankfully, these were largely resolved, but there are some other issues we found when comparing the content indexed on Google and that on Bing.
Google’s indexing
Despite many lingering indexing issues, Google has made some adjustments to MarTech’s indexation over the past year. The search engine removed virtually all of our duplicate URLs after we set up our redirects, and a good portion of Marketing Land and MarTech Today pages have been removed as well. However, we’ve recently noticed some interesting performance and indexing trends.
Performance. The majority of the top pages from the past three months in terms of interaction are legacy pages that have little to no relevance to our MarTech brand. Aside from the homepage, the “What is MarTech” page, and our CDP platform page, the top URLs are largely irrelevant to our target audience.
Granted, these articles have been live for years, building up authority on the Marketing Land and MarTech Today domains. But, after almost a year of MarTech being live, it’s odd that there are so many old, less relevant pages sitting at the top of our performance lists – especially when our team has published so much good content since then.
Indexed pages. Google has roughly 29,000 MarTech URLs in its index. The majority of these are relevant links we’ve placed in our sitemaps. However, there are over 7,000 URLs in the “Indexed, not submitted in sitemap” category. Many of these URLs are irrelevant — a disconcerting number have parameters that look like either tracking code or, in some cases, spam.
URLs with parameters in the index.
The prevalence of URL parameters isn’t surprising, but it’s not clear why Google is including so many of these in the index. The more alarming trend, however, is the number of Marketing Land and MarTech Today URLs that are still in Google’s index as well.
Marketing Land URLs on Google.
MarTech Today URLs on Google.
We know that there are plenty of Marketing Land and MarTech Today URLs online, both in our older pieces of content and on other websites. But it’s strange to see so many still in Google’s index.
Get the daily newsletter search marketers rely on.
Bing’s indexing tells a different story. Though there are still plenty of irrelevant content pieces, they’re much less prominent in the SERPs.
Performance. MarTech’s top-performing pages on Bing look somewhat similar to those on Google. The homepage, “What is MarTech” page, and legacy pages are still there, but we also found one of our more recent news articles in the mix. The importance of the piece to our industry undoubtedly helped bring it to the forefront, but it’s peculiar that Google didn’t treat it the same way.
This newer article’s numbers are encouraging, but, just like the results on Google, our more relevant topic pages are failing to perform well.
Indexed pages. Bing has indexed fewer of our MarTech pages (roughly 17,000 URLs), which isn’t surprising, given how much smaller it is than Google. However, after analyzing these URLs, we found the ratio of relevant content to irrelevant content to be much lower. We’re not seeing a huge number of indexed URLs with parameters.
The most glaring difference between the two search engines is their indexing of our old domain pages. While Google still retains over 2,000 URLs from Marketing Land and MarTech Today, there are only 143 of these URLs left in Bing’s index.
Marketing Land URLs on Bing.MarTech Today URLs on Bing.
Yes, Bing had fewer of these pages to begin with, but the inconsistency is still shocking.
A discrepancy between Google and Bing’s indexing
Of the two search engines, it seems Bing is doing a better job of crawling our old URLs and adjusting its index accordingly. This makes sense — there are fewer pages indexed on Bing, so the search engine has less to clean up.
But why is Google holding on to so many of these old URLs? One possible explanation is that it simply hasn’t crawled all of the old URLs yet. This would mean it hasn’t found the 301 redirects we put in place, believing the old sites are still live.
This seems unlikely, however, as we migrated the site almost a year ago. Google has had plenty of time to crawl our pages. Yet, we’re still open to this possibility.
Another explanation could be that there’s a structural issue on the MarTech site that is somehow telling Google the old domains are still live. We’re conducting some deep technical audits at the moment to determine if this is true. Until we know more, we’re going to continue to create good content and do all we can to help it rank higher than the less relevant pages.
Have you noticed discrepancies in indexing between Google and Bing? How are you addressing the issue? Email me at cpatterson@thirddoormedia.com with the subject line “Messy SEO Part 7” to let me know.
More Messy SEO
Read more about our new MarTech domain’s SEO case study.
Microsoft Bing’s search results for queries related to the conflict in Ukraine show an Opinions section and two timelines (one in the main column and another in the knowledge panel).
Microsoft Bing’s search results page for the query [Ukraine Russia]. Click to expand.
The Opinions section. On Bing, news results typically appear in the “News about” section (at the top of the example above; this feature is similar to Google’s Top stories section) or in the standard, non-rich result listings.
However, media outlets often publish opinions and commentary on current events from non-staff writers. Bing has started highlighting these types of results in their own section, “Opinions.”
The Opinions section in Bing search results.
For the query [Ukraine Russia], the Opinions section included six results from different media outlets. When a featured image isn’t available, Bing will show a preview of the article (as shown above).
The timelines. For associated queries, Bing may show up to two timelines: the horizontal timeline that appears in the main column of search results and the vertically oriented timeline within the knowledge panel (on the right-hand rail of the search results).
The timeline within the knowledge panel begins on August 24, 1991, when Ukraine became an independent state following the dissolution of the Soviet Union, and includes prior events, such as the annexation of Crimea, that have led to the current state of Russo-Ukrainian affairs.
The horizontal timeline in the main column only shows events from February 9 until March 5 (the screenshot below was taken on March 14).
The Timeline feature in Bing’s main search results column.
Additionally, the main column timeline cites sources, whereas the knowledge panel timeline does not.
Why we care. The Opinions section may help Bing contextualize some news-related results for users. Informing users that a result is an opinion (and not strictly news) helps them understand the nature of the content, thus potentially reducing misinformation.
Likewise, the timeline features may help users understand how an event has unfolded, which may also help them make sense of the content on that same search results page. For example, if a timeline were present on coronavirus-related search results, it might provide a frame of reference for users, enabling them to better interpret the number of daily new cases or understand whether restrictions are being loosened or heightened.
The visibility opportunities associated with these features are primarily for news publishers, but as was the case with Google’s COVID-related search results page, these features may eventually receive a wider rollout, which could open up opportunities for brands as well.
Microsoft announced that IndexNow is now also supported by another WordPress plugin named SEOPress. This gives another 200,000 sites that are using this plugin easy access to enable IndexNow on their WordPress sites.
“By integrating the possibilities offered by the API, SEOPress allows more than 200,000 sites to be indexed more quickly and efficiently, all without any technical knowledge necessary and automatically,” Microsoft said.
IndexNow growth. Just last week, IndexNow has grown to support millions of sites by adding integration with Duda, All in One WordPress SEO plugin and Rank Math SEO plugin. Last November, Cloudflare announced it added integration with IndexNow and 60,000 websites turned on that feature. Microsoft released a simple to use WordPress plugin for IndexNow and Google announced it would be testing this protocol. Then last month, Microsoft said 80,000 websites are now using IndexNow for URL submission. With these providers integrating IndexNow, we are now up to millions of sites adopting IndexNow, just several months after IndexNow was initially rolled out.
What is IndexNow. IndexNow provides a method for websites owners to instantly inform search engines about latest content changes on their website. IndexNow is a simple ping protocol so that search engines know that a URL and its content has been added, updated, or deleted, allowing search engines to quickly reflect this change in their search results. The co-sharing of IndexNow went fully live last month in January 2022.
How to activate IndexNow in SEOPress. First, you need version 5.4 of SEOPress, then to enable IndexNow, check off the “automatically notify search engines” option is checked from the settings tab of the plugin.
How it works. If you using a provide or Plugin that has this built in, just follow the steps they provide. If you need to code this manually into your CMS, then the protocol is very simple — all you need to do is create a key on your server, and then post a URL to the search engine to notify IndexNow-participating search engines of the change. The steps include:
Host the key in text file named with the value of the key at the root of your web site.
Start submitting URLs when your URLs are added, updated, or deleted. You can submit one URL or a set of URLs per API call.
Why we care. Like we said before, instant indexing is an SEO’s dream when it comes to giving search engines the most updated content on a site. The protocol is very simple and it requires very little developer effort to add this to your site, so it makes sense to implement this if you care about speedy indexing.
Microsoft continues to work with providers to adopt IndexNow and grow its reach to millions of sites.
Microsoft made a few announcements this week on the Bing webmaster blog around more software providers adding support for IndexNow leading to millions of sites using IndexNow since its launch in October 2021. Part of these announces have that Duda, All in One WordPress SEO plugin and Rank Math SEO plugin all added IndexNow integration into their systems.
What is IndexNow. IndexNow provides a method for websites owners to instantly inform search engines about latest content changes on their website. IndexNow is a simple ping protocol so that search engines know that a URL and its content has been added, updated, or deleted, allowing search engines to quickly reflect this change in their search results. The co-sharing of IndexNow went fully live last month in January 2022.
Millions of sites using IndexNow. Last November, Cloudflare announced it added integration with IndexNow and 60,000 websites turned on that feature. Microsoft released a simple to use WordPress plugin for IndexNow and Google announced it would be testing this protocol. Then last month, Microsoft said 80,000 websites are now using IndexNow for URL submission. We are now up to millions of sites adopting IndexNow, just several months after IndexNow was initially rolled out.
New adoption. In these three announcements, we had the number 20 CMS platform on the internet, Duda announce they have enabled IndexNow by default across nearly 1 million active websites. The All In One WordPress SEO plugin announced it added support for IndexNow to the over 2 million sites that use its plugin. To activate and use IndexNow with AIOSEO, users can head over to the Feature Manager tab in the AIOSEO dashboard, where they’ll find the option to activate the addon. Then Rank Math SEO plugin announced it added support to the 1.1 million sites that use their plugin. Check out those individual blog posts to learn how to activate IndexNow if you use those tools.
Get the daily newsletter search marketers rely on. See terms.
How it works. The protocol is very simple — all you need to do is create a key on your server, and then post a URL to the search engine to notify IndexNow-participating search engines of the change. The steps include:
Host the key in text file named with the value of the key at the root of your web site.
Start submitting URLs when your URLs are added, updated, or deleted. You can submit one URL or a set of URLs per API call.
Why we care. Like we said before, instant indexing is an SEO’s dream when it comes to giving search engines the most updated content on a site. The protocol is very simple and it requires very little developer effort to add this to your site, so it makes sense to implement this if you care about speedy indexing.
Now with these three new software players adding support, it makes adoption even easier for millions of sites.
On February 8, 2022, Google released a document to the U.S. Copyright Office saying “when a site is demoted [by the Pirate update], the traffic Google Search sends it drops, on average, by 89% on average.” This is a statement about Google’s efforts to remove sites that “received a large number of valid removal notices” as DMCA requires, hence the Google Pirate update from 2012.
The Pirate update. The Pirate update, which Google originally called the DMCA update, looked at if a site had a large number of DMCA takedown requests and if so, it demoted the site. Google officially only confirmed updating this algorithm once after its launch in 2012, that was in 2014. It is without a question that Google runs this algorithm update periodically to catch new sites that may be copying copyrighted content from others. But Google clearly does not announce each time the search company runs this update.
Google said in the new PDF document “we have developed a ‘demotion signal’ for Google Search that causes sites for which we have received a large number of valid removal notices to appear much lower in search results.”
89% demotion. This PDF document written by Google states that when sites are demoted by this algorithm, the sites hit on average see 89% less Google Search traffic as a result. The document states “When a site is demoted, the traffic Google Search sends it drops, on average, by 89% on average.”
Redirect tricks. Google also said that it has a flag named “still-in-theaters/prerelease” that will pick up on when a site hit by this update redirects to a new domain without that flag. Google said, “We have also made it much harder for infringing sites to evade demotion by redirecting people to a new domain.” Google added “we have added a “still-in-theaters/prerelease” flag for DMCA notices involving this category of content to enhance the Search demotion signal.”
This report comes via TorretFreak, a publication that tracks the latest news about copyright, privacy and related topics. Also, a hat tip to @GlennGabe for notifying me of this.
Why we care. If a site gets too many (how many is unknown) DMCA takedown requests, it can lead to that site being hit by this Pirate update. If a site gets hit by the Pirate update, you can expect, on average, 89% less traffic from Google Search to that site. And no, redirecting that site to a new domain name won’t seem to help you.
So if you run into any clients during an audit that have these notices, which you might see Search Console notification, then maybe check the Google transparency report for the domain and then maybe request reconsiderations after cleaning up the issues.
JavaScript frameworks have been growing in popularity over the last few years, thanks in no small part to the flexibility they offer. “JavaScript frameworks allow for rapid development. It offers better user experience. It offers better performance and it offers enhanced functionality that traditional frameworks — non-JavaScript ones — sort of lack,” said Nati Elimelech, tech SEO lead at Wix.
“So, it’s no surprise that very large websites or complex UIs with complex logic and features usually tend to use JavaScript frameworks nowadays,” he added.
At SMX Next, Elimelech provided an overview of how JavaScript works for client-side, server-side and dynamic rendering, and shared insights for auditing gained from implementing JavaScript on over 200 million websites.
Client-side vs. Server-side rendering
Different rendering methods are suitable for different purposes. Elimelech advocated on behalf of dynamic rendering as a means to satisfy search engine bots and users alike, but first, it’s necessary to understand how client-side and server-side rendering work.
Client-side rendering
When a user clicks on a link, their browser sends requests to the server that site is hosted on.
“When we’re talking about JavaScript frameworks, that server responds with something that’s a bit different than what we’re used to,” Elimelech said.
“It responds with a skeleton HTML — just the basic HTML, but with a lot of JavaScript. Basically, what it does is tell my browser to run the JavaScript itself to get all the important HTML,” he said, adding that the user’s browser then produces the rendered HTML (the final HTML that is used to construct the page the way that we actually see it). This process is known as client-side rendering.
Image: Natie Elimelech.
“It’s very much like assembling your own furniture because basically the server tells the browser, ‘Hey, these are all the pieces, these are the instructions, construct the page. I trust you.’ And that means that all of the hard lifting is moved to the browser instead of the server,” Elimelech said.
Client-side rendering can be great for users, but there are cases in which a client doesn’t execute JavaScript, which means it won’t get the full content of your page. One such example may be search engine crawlers; although Googlebot can now see more of your content than ever before, there are still limitations.
Server-side rendering
For clients that don’t execute JavaScript, server-side rendering can be used.
“Server-side rendering is when all of that JavaScript is executed on the server-side. All of the resources are required on the server-side and your browser and the search engine bot do not need to execute JavaScript to get the fully rendered HTML,” Elimelech explained. This means that server-side rendering can be faster and less resource-intensive for browsers.
Image: Natie Elimelech.
“Server-side rendering is like providing your guests with an actual chair they can sit it on instead of having to assemble it,” he said, continuing his previous analogy. “And, when you do server-side rendering, you basically make your HTML visible to all kinds of bots, all kinds of clients . . . It doesn’t matter what the JavaScript capabilities are, it can see the final important rendered HTML,” he added.
Dynamic rendering
Dynamic rendering represents “the best of both worlds,” Elimelech said. Dynamic rendering means “switching between client-side rendered and pre-rendered content for specific user agents,” according to Google.
Below is a simplified diagram explaining how dynamic rendering works for different user agents (users and bots).
Image: Natie Elimelech.
”So there’s a request to URL, but this time we check: Do we know this user agent? Is this a known bot? Is it Google? Is it Bing? Is it Semrush? Is it something we know of? If it’s not, we assume it’s a user and then we do client-side rendering,” Elimelech said.
In that case, the user’s browser runs the JavaScript to get the rendered HTML, but still benefits from the advantages of client-side rendering, which often includes a perceived boost in speed.
On the other hand, if the client is a bot, then server-side rendering is used to serve the fully rendered HTML. “So, it sees everything that needs to be seen,” Elimelech said.
This represents the “best of both worlds” because site owners are still able to serve their content regardless of the client’s JavaScript capabilities. And, because there are two flows, site owners can optimize each to better serve users or bots without impacting the other.
But, dynamic rendering isn’t perfect
There are, however, complications associated with dynamic rendering. “We have two flows to maintain, two sets of logics, caching, other complex systems; so it’s more complex when you have two systems instead of one,” Elimelech said, noting that site owners must also maintain a list of user agents in order to identify bots.
Image: Natie Elimelech.
Some might worry that serving search engine bots something different than what you’re showing users can be considered cloaking.
“Dynamic rendering is actually a preferred and recommended solution by Google because what Google cares about is if the important stuff is the same [between the two versions],” Elimelech said, adding that, “The ‘important stuff’ is things we care about as SEOs: the content, the headings, the meta tags, internal links, navigational links, the robots, the title, the canonical, structured data markup, content, images — everything that has to do with how a bot would react to the page . . . it’s important to keep identical and when you keep those identical, especially the content and especially the meta tags, Google has no issue with that.”
Image: Natie Elimelech.
Since it’s necessary to maintain parity between what you’re serving bots and what you’re serving users, it’s also necessary to audit for issues that might break that parity.
To audit for potential problems, Elimelech recommends Screaming Frog or a similar tool that allows you to compare two crawls. “So, what we like to do is crawl a website as Googlebot (or another search engine user agent) and crawl it as a user and make sure there aren’t any differences,” he said. Comparing the appropriate elements between the two crawls can help you identify potential issues.
Image: Natie Elimelech.
Elimelech also mentioned the following methods to screen for issues:
Visual inspection by switching the user agent in the browser and/or turning off JavaScript to see if anything changes between the versions.
Google Search Console can be used to see what kind of HTML is returned to Google and how it can render it.
“Remember, JavaScript frameworks aren’t going anywhere,” he said. “Chances are you’re going to meet one of them soon, so you better be prepared to handle them.”
Microsoft has moved the Bing News PubHub out of its own microsite and into the overall Bing Webmaster Tools console, the company announced this morning. This makes it easier for for publishers who have already verified their sites with Bing Webmaster Tools to access the Bing News PubHub features.
Bing News PubHub. Bing News PubHub first launched in June 2016 as a portal for news publishers to submit their sites, at pubhub.bing.com. While pubhub.bing.com is still live, it is now telling publishers “We have integrated with Bing Webmaster, please go to Bing Webmaster Tool.”
What it looks like. Here is a screenshot of my site, the Search Engine Roundtable, waiting to for approval in Bing News. Note, you can access Bing News PubHub on the left side navigation in Bing Webmaster Tools:
How Bing evaluates news sites. Bing News uses the following criteria to help evaluate the news websites:
Newsworthiness – Reporting on timely events and topics that are interesting to the people using our services. Content that does not focus on reporting, such as how-to articles, job postings, advice columns, product promotions, is not considered newsworthy. Similarly, content that consists strictly of information without including original reporting or analysis, such as stock data and weather forecasts, is not considered newsworthy.
Originality – This includes unique facts or points of view. Faced with numerous sources frequently reporting similar or identical content, originality or uniqueness becomes a critical way to determine the value to a user of an individual story.
Authority –News sites with authority maintain the highest level of trust and respect from our users. Submissions must identify sources, authors, and attribution of all content.
Readability –Content containing correct grammar and spelling and that keeps site design easy for people to navigate. Advertising should never interfere with the site experience.
Why we care. If you are a publisher and want to see the status of your approval in Bing News, you can now login to Bing Webmaster Tools, click on the Bing News PubHub and check your status there. If you have not yet submitted your site for approval, you can within that tool as well. If you were previously rejected, you can resubmit the site for an additional review.
This guide was first published on May 21, 2020. It was updated to include new information on February 4, 2022.
Optimizing your Shopify store for Google, Bing and other search engines is critical for helping potential customers discover your site. Many of the tenets of search engine optimization (SEO) are based on providing users with a positive experience. The easier it is to discover and use your Shopify store, the more likely people are to shop with you.
The fundamentals covered in our SEO guide also apply to Shopify SEO, but there are some tricks involved in getting a Shopify site fully optimized for search. There are also nuances and other factors to be mindful of when evaluating the platform’s SEO capabilities.
In this guide, we address the SEO fundamentals and technical considerations to improve your Shopify store’s search rankings to increase traffic and sales.
Shopify SEO basics
Shopify’s page editor. Shopify’s content management system (CMS) has a page editor with a simple user interface to build landing pages, collections pages, product detail pages and blog posts.
Shopify’s page editor.
The page editor allows for basic formatting options, headings, tables and the ability to edit in HTML. You can also upload photos and embed videos to the pages.
The search engine listing preview section automatically generates a page title and meta description based on the content you enter in the page details section. By default, your page title is used for the title tag and H1 and your description is used for the meta description. Site owners can overwrite these automated defaults by modifying their search engine preview section.
Page titles are an important ranking factor. They provide search engines with information about the contents of your page and they often appear as the headline for your search result.
Meta descriptions are often what appear under the title in search results. They should summarize the content of the page for prospective customers, and can help improve click-throughs to your site.
The Shopify page editor allows you to manually revise your titles and descriptions. If you’re not content with the ones automatically generated, optimize them to include the keywords you’d like to rank for and appeal to customers browsing the search results.
Analytics. Shopify sites are compatible with two very useful, free tools: Google Search Console and Google Analytics. Google Search Console helps site owners monitor and troubleshoot their site’s presence in Google search results, and provides data on the queries that are leading prospects and customers to your pages. Google Analytics shows you how people arrived at your site and behaved once they got there. Search Console and Analytics are critical tools for SEO.
To connect your Shopify store to Google Search Console, you’ll need to add a piece of code to your homepage (Shopify has a great video on this). There are two things to keep in mind when verifying your Search Console account for your Shopify store: Google will only index the current theme and only if the theme is live and not password protected (new merchants often password-protect their sites because they’re not yet ready for launch).
[Pro Tip]Use Shopify’s URL structure to help segment your analysis
Thanks to Shopify’s strict URL structure, it’s easy to segment certain types of content in Google Search Console, Google Analytics, etc. For example, if you want to see how your products are performing, simply filter your data by URLs that contain “/products/” and voila! The same trick works for collections (/collections/), blog posts (/blogs/), and pages (/pages/).
-Kevin Wallner, founder, First Chair Digital
Domain and security. You can buy a custom domain through Shopify; the platform even has a free tool to check domain name availability and generate alternative domain names. If you don’t want a custom domain, your site’s URL will be yoursitename.myshopify.com. Having your own custom domain is much better branding, conveys more professionalism and greatly increases your chance of ranking well in searches for your business’ name.
If you opt for a custom domain, check that your .myshopify.com domain redirects to your primary domain by logging into your Shopify account and heading to Settings > Domains (within the left-hand navigation menu). Your primary domain should say “Traffic from all your domains redirects to this primary domain.” If it does not, click on “Enable redirection.” Doing this will help to ensure that search engines only show your primary domain to searchers, instead of presenting both the primary and the .myshopify.com domain.
All Shopify plans also come with an SSL certificate, which encrypts data sent between your site and its visitors. This is important because Google gives a small ranking boost to secure sites and because many browsers will display a security warning when users attempt to visit a site without an SSL certificate. You can check this one off your worry list.
Site speed. Site speed and security are Google ranking factors, and they go hand-in-hand with user experience. Slow load times may lead customers to bounce before your page loads, which means they won’t be buying from you.
Selecting a lightweight theme for your Shopify store can help you get a head start on your speed optimizations. Loading times can vary across themes, said Chris Long, director of e-commerce at Go Fish Digital, during our Shopify SEO session of Live with Search Engine Land.
His agency found that the average theme takes approximately 3.8 seconds for First Contentful Paint (FCP), makes 171 requests and weighs in at about four megabytes. If you’ve already selected a Shopify theme, those benchmarks can help you gauge whether your store is performing below or above the mean.
Shopify features a content delivery network (CDN) that can help keep page load times down no matter where in the world your customers are browsing. Even so, it’s in your best interest to compress your images (more on that below) and implement lazy loading functionality to shave down your loading times as much as possible.
Instead of loading all your images the instant a visitor lands on your page, lazy loading enables the visitor’s browser to load images as they scroll, decreasing the upfront load time. “It’s actually exceptionally easy for developers to implement in most cases,” Long said.
Being conservative with your Shopify apps (more on that below) can also help you manage page load times. And, tools such as Google’s PageSpeed Insights can help you identify more opportunities to increase site speed by suggesting different image formats and other speed optimizations.
Selecting the appropriate format to compress your images is one way to keep file sizes under control: JPEGs will generally result in smaller file sizes than PNGs, which are more suitable when you need a transparent background in your image.
Every image should have an alt attribute assigned to it. Image alt text provides search engines with more context as to what the page is about, which can improve relevance and thus rankings. Additionally, the alt text is read aloud for site visitors that rely on screen readers, so adding it will help you comply with digital accessibility standards.
From your Shopify admin, alt text can be added by navigating to the desired product detail page and clicking the desired image or media item. A media preview page will open and you can click “Add alt text” to edit the field. Once you’re done, save the alt text and exit the preview page.
Store owners can use image optimization apps (more on Shopify apps in the next section) to automate some of these tasks. TinyIMG SEO Image Optimizer has hundreds of five-star reviews and can help keep your load times down by compressing your images.
Shopify does provide an image sitemap, which helps Google find your images so that they may potentially appear in image search results and provide searchers with one more way to discover your products. However, Shopify’s image sitemap only includes one image per product and does not include additional metadata. To get more of your images indexed, Kevin Wallner, founder of First Chair Digital, recommends the Image Sitemap app, which can build, submit, monitor and update XML sitemaps for all the images in your store.
Shopify apps. One big advantage of a platform like Shopify is the app ecosystem that makes it easy to add more features and tools to your site with little to no coding. There are apps to help you manage inventory, take customer support tickets, run affiliate programs and, yes, optimize your site for search.
Shopify has dozens of SEO apps, including Yoast SEO, which offers much of the same functionality as its well-known WordPress counterpart. Yoast SEO can help you control your titles and descriptions in Google Search and social media and provide feedback on readability. It can also help you implement structured data, which can come in handy as structured data is used to power product information in image search results, which may, in turn, help you guide searchers from Google’s image results directly to your product page.
In 2021, both Google and Bing launched Shopify integrations. These integrations are actually apps (the Google Channel and Microsoft Channel apps) and they both allow merchants to include their products in Google’s and Bing’s free product listings. This is a welcome addition for Shopify retailers since these integrations make it easier for them to get greater search visibility without necessarily needing to rely on an expert. And, both apps can also be used to launch shopping campaigns, in case you want to advertise your offerings.
There are also plenty of other apps that can help you do things like add product reviews to give customers more information about what you’re selling, which search engines may use to show star ratings that can improve your clickthrough rates.
Before you load up on apps, keep in mind that they add extra code to your site. Apps that load on the user’s end, such as some that enable you to customize your store’s design, may slow the site down.
Sitemaps. Sitemaps contain information about your pages and files and are used by search engines to crawl and index your site — this process enables your pages to appear in search results. Shopify automatically generates a sitemap.xml file, with links to your pages, products, images, collections and blog posts, for all sites. After verifying your Google Search Console account, submit your sitemap to Google to help it find and index your pages.
Crawl optimization. For merchants that want to control which pages get crawled, Shopify now lets site owners edit their robots.txt files. This can be used to disallow certain URLs from being crawled, add extra sitemap URLs, block crawlers and so on.
International domains. In March 2021, Shopify launched international domains, enabling merchants to sell to more than one country using a single account. Merchants can now configure subdomains and language folders for international targeting. When you do so, Shopify automatically adds hreflang tags to your store, which can help save you time.
Local SEO
Since the onset of the COVID pandemic, more local merchants have made their way to Shopify. If your business serves a locale, ensure that you have an online presence wherever customers may go to look for businesses like yours, be it Google, Bing, Facebook or other platforms.
While your Shopify website will complement your local efforts, the range of options for local search optimization warrants numerous standalone articles. As a jumping-off point, check out our how-to guide on optimizing your Google local knowledge panel.
Technical SEO for Shopify
Search engines work by “crawling” websites, meaning that they look through a site’s code and URLs to discover each page on the site. That information is then added to the search engine’s index to be ranked and served as a result when someone conducts a relevant search.
Technical SEO refers to the optimizations that facilitate the crawling and indexing aspects of search. Technical optimizations can include your site’s architecture, URL structure and JavaScript.
Shopify enables store owners to bypass many of the technical aspects of launching an e-commerce site by providing pre-made themes with Shopify’s framework as the structure. However, that structure is rigid and in some instances, does not provide a straightforward way to make certain technical optimizations.
Being aware of the technical SEO challenges inherent to Shopify can help you find workarounds for a number of issues and inform prospective store owners about the trade-offs that come with building their store on Shopify.
Below are the most common technical SEO challenges that Shopify store owners encounter. Solutions to these issues are discussed in our Technical SEO for Shopify guide.
Duplicate pages. Collections help group products together into categories, making it easier for customers to find them. They’re also an inherent property of the Shopify framework.
When you associate a product page with a collection page (as the vast majority of merchants are likely to do), Shopify will generate a second URL for that product page. For some of the free themes, this means you’ll have two URLs for the same product. The URLs you end up with will look something like this:
Duplicate pages can split your link-building power as referrers may link to either URL. Duplicate content can also make it harder for search engines to determine which URL to index and rank. While Shopify has remedied this issue for some of its free themes, canonicalizing the duplicate, some themes may still be affected, so we’re leaving this guidance in place, just in case.
Internal linking. The duplicate pages issue mentioned above can also show up in your internal links. In the image below, Shopify uses two different links to direct customers to the same product detail page.
The left image shows the link for a face mask as it appears on the store’s homepage. The right image is the link for the same face mask as it appears in the recommended products section of another product page.
While internal links don’t directly play into Google’s algorithms, they do direct link equity to your product detail pages. That link equity gets diluted when it is spread across multiple links and search engines will have a harder time understanding which one is the primary URL.
Architecture issues. Shopify automatically generates the URL for your product detail pages using the following structure: myshopifystorename.com/products/product-name. Store owners can only modify that last part of the URL, where the product’s name (which is derived from the page title) appears.
Shopify automatically generates URLs for product and collection pages. Store owners can only modify the last part of the URL (indicated in green).
URLs are a minor ranking factor, so you’ll want to use descriptive words in your URLs to inform search engines about your page, but also to give potential customers an idea of what they’re clicking through to. Removing the “/products” or “/pages” URL paths gives you more space to include those descriptive keywords; unfortunately, Shopify doesn’t have that option out-of-the-box. There are, however, solutions, such as Cloudflare and their Cloud Worker interface, that can be implemented with the help of a developer, as discussed in our Technical SEO for Shopify guide.
Product schema. Schema is a type of structured data that can help search engines understand your site’s content. Search engines also use schema to generate rich snippets, which can provide your products with greater visibility in the search results.
Bombas, a Shopify store, uses schema to indicate that a product is in stock.
Google’s Popular Products section is another search feature that uses schema to provide potential customers with product information, although it is currently only available for apparel and fashion items.
With an app like Feed for Google Shopping or ShoppingFeeder, you can sync your entire product inventory with Merchant Center. Once you’re in Merchant Center, if you enable ‘Local Surfaces Across Google’ in Programs, your products can instantly appear in Maps results when people search product keywords . . . Surfaces Across Google creates a Google-hosted storefront, so you might see a decline in visitors in your Shopify analytics if customers are browsing products on Google instead; however, these visitors can be tracked in Merchant Center.
–Lachlan Wells, Berlin-based SEO consultant
Product schema is built into most Shopify themes. The default theme, “Dawn,” supports the following schema types: “Organization,” “WebSite,” “Article” and “Product.”
However, there may be additional structured data types, such as recipe or FAQ schema, that you’d like to add but are not supported by your theme. In that case, you can edit the structured data code within your theme, dynamically inject structured data using JavaScript or install a dedicated schema app. Whichever method you choose, you can verify if your pages are marked up correctly using Google’s Structured Data Testing Tool.
SEO, beyond products and category pages
Creating valuable, informative and engaging content can help increase your store’s organic visibility. Content can give search engines more information about your site and prospective customers more ways to discover your business. More visibility can mean more traffic and more sales.
The pages every store should have. It’s highly recommended that you create an “About us” and “Contact” page; these pages help potential customers get familiar with your business and enable them to get in touch with you to resolve concerns before and after they buy from you.
Additionally, a dedicated page for your shipping information, return policy, privacy policy, terms and conditions and frequently asked questions can also enable you to communicate important details and may even reduce the amount of time spent answering redundant questions. You can also add a mission statement or “Our Story” page to help distinguish your brand from others.
Keyword research. Keyword research helps you understand how your target audience searches — for the kinds of products you sell, the types of problems they’re looking to solve and other queries that are relevant to your business. You can then use these words to inform your content strategy and optimize your pages.
There are a number of keyword research tools that help indicate how popular queries are, seasonal trends and related terms. Google Trends analyzes search query interest, which can be filtered by time and geographic region, displays related queries and allows you to compare interest between different keywords. You can also examine the auto-suggested queries that populate in the search bar on Google, Bing and Amazon to get more keyword ideas.
Google Ads and Microsoft Advertising offer in-depth keyword tools that include monthly search volume and competition estimates and suggested bid amounts. Google Keyword Planner and Microsoft Keyword Planner are designed for advertisers, but they are both free to use and provide a wealth of information.
You’ll also want to evaluate what the search results look like for the keywords you’re researching. The “People also ask” box and the “Searches related to” section can supplement your keyword research.
Take a look at the search results and features present on the page to get an idea of what search engines are surfacing for that set of terms. Are the results filled with news articles, signaling an informational intent, or are they businesses selling products similar to yours? If it’s the latter, you have a higher chance of ranking for those keywords.
E-commerce content ideas. Providing your audience with pertinent content can help them get more value from your products or services while also signaling relevance to search engines. Here are a few content ideas that might help you get started:
How-to’s and tutorials can inform your customers about how your products or services work. Consider creating a video to demonstrate these points; you can then upload it to YouTube and embed it on your blog post or product detail page to encourage visitors to shop with you.
Behind-the-scenes content can bolster your branding and give customers a reason to shop with you over competitors. For example, if you sell vegan cookie dough, you could create a video that demonstrates how you test new recipes to teach customers about your process while getting them excited for upcoming flavors.
Case studies can separate your business from a crowded field of competitors. If you’re selling tie-dye t-shirts, for example, you could create a video detailing how your shirts’ colors hold up against competing brands, wash after wash.
Looking for more ways to optimize and market your Shopify store? Check out these resources:
Microsoft Bing now has new car and automobile search features to let you find your next car, the company announced on its blog. You can search for used cars on Bing, you can search for specific car makes and models and then Bing will provide you search box that you can filter to find the car you are looking for.
What it looks like. Here are some screenshots of the search box in Microsoft Bing for searches on Kia K5 and SUV for sale:
Also, there are links to the MSN Autos marketplace in the knowledge panels for these car related searches under the “learn more” and “shop used cars” right above the car specifications.
Local. You can also browse local car inventory on Bing Local and Bing Maps. Microsoft said you “can now quickly and easily find vehicles for sale on Bing.com’s new Local Auto Dealership Storefront or using the browse-by-map feature.”
List your car. Microsoft allows you to list your car on MSN autos over here, where you fill out a listing form that will post your vehicle ready for sale. “In just a few simple steps, sellers enter the basics about the car for sale, upload photos, add a detailed description and finish the listing with any extra info that might make the car appealing to buyers. Your car is now for sale,” Microsoft said.
Why we care. Buying a car these days, in this market, is incredibly hard – there is very little inventory, what is available is very expensive and hard to get your hands on. Maybe this will make it easier for you to find the car you are looking for and also make it easier to sell your used car faster.
