20211008 SEL Brief
The post 20211008 SEL Brief appeared first on Search Engine Land.
Google will introduce a new policy for advertisers, AdSense publishers and YouTube creators that prohibit ads for, and monetization of, content that contradicts authoritative scientific consensus around the existence and causes of climate change, the company announced Thursday. Google will use its automated systems as well as human reviewers to enforce the policy, which will take effect next month.
Why Google is making this change. “In recent years, we’ve heard directly from a growing number of our advertising and publisher partners who have expressed concerns about ads that run alongside or promote inaccurate claims about climate change,” the company said, “Advertisers simply don’t want their ads to appear next to this content. And publishers and creators don’t want ads promoting these claims to appear on their pages or videos.”
What kind of content and ads are prohibited? The new policy prohibits ads for, and monetization of, the following: content that refers to climate change as a scam or a hoax, claims denying that long-term trends show the global climate is warming and claims denying that greenhouse gas emissions or human activity contribute to climate change.
Sorting out offending content and ads. “We’ll look carefully at the context in which claims are made, differentiating between content that states a false claim as fact, versus content that reports on or discusses that claim,” Google said in its announcement, “We will also continue to allow ads and monetization on other climate-related topics, including public debates on climate policy, the varying impacts of climate change, new research and more.”
Why we care. Advertisers that deal with climate matters should be especially careful as this new policy takes effect. If your ads adhere to the policy but are still disapproved, you can appeal the policy decision directly from your Google Ads account.
Publishers and YouTube content creators that make climate change content should also be careful not to run afoul of this policy as it may make monetization impossible.
For general publishers and YouTube creators that monetize their content, this change may increase brand safety as climate-change-denying ads will be less likely to show up alongside your content.
The post Google takes stand against climate change deniers with new ads and monetization policy appeared first on Search Engine Land.
With the iOS 15 release on September 20th, AMP URLs in Google Search just stopped working. Google is aware of the issue and said the issue should be resolved soon. This is only impacting Google searchers using Google Search on iOS devices that have been upgraded to iOS 15.
What happens. If you are on iOS 15 and you click on a website listing in the Google Search results, you will not be taken to an AMP URL, even if the AMP URL exists for that listing. Instead, you will likely be taken to the main site’s URL for that page, or a link to their mobile app – if you have the app pre-installed on your device.
Jeff Johnson, an iOS developer, dug into this and said with the iOS 15 Safari “User-Agent, there are no AMP links in Google search results, but if I simply change Version/15.0 to Version/14.0 and keep the rest the same, Google search results suddenly have AMP links again!” “This is reproducible on my iPhone, in the Xcode iPhone simulator, and also in desktop Safari Mac with its User-Agent spoofed as iPhone,” he added.
I added an example of what this looks like on my personal blog, if you want to see it in action.
Confirmed bug. Danny Sullivan of Google confirmed the bug on Twitter saying “It’s a bug specific to iOS 15 that we’re working on. We expect it will be resolved soon.”
Why we care. You might notice weirdness with your analytics data around your AMP URLs coming from Google Search over the past couple of weeks. Your traffic should not be impacted because the traffic is still going to your main URL, but traffic to your AMP URLs may have a decline due to this bug with iOS 15. It is still not yet resolved but it should be resolved soon.
The post AMP links in Google Search break with iOS 15 appeared first on Search Engine Land.
Google will move AdSense from a second-price auction model to a first-price auction by the end of 2021, the company announced Thursday. There is no action for advertisers or publishers to take and these changes will occur automatically.
First-price vs. second-price auctions. In a second-price auction, the final price paid by the winner is determined by the second-highest bid. In a first-price auction, the final price is the same as the winning bid.
This will simplify things for advertisers, Google says. “On display ad selling platforms, a first-price auction simplifies the buying experience for advertisers because the final price reflects the winner’s bid,” Google said on its FAQ page about AdSense moving to a first-price auction.
Which AdSense products are affected. The transition to a first-price auction only affects AdSense for Content, AdSense for Video and AdSense for Games. It does not affect AdSense for Search or AdSense for Shopping.
Why we care. Transitioning to a first-price auction aligns AdSense with Ad Manager and AdMob, both of which are already operating under that model. Making the winning bid the actual price advertisers pay may make it easier for some advertisers to plan their spending.
However, a first-price auction means that the final price the winning advertiser pays will typically be higher than it was under a second-price model. Campaign managers should inform stakeholders of this change as it may affect their budget.
Google also said that publishers will likely not see a change in their earnings as a result of this transition: “Due to the dynamic auction environment, we cannot predict how specific AdSense publishers will be impacted. But, on average we expect the impact to AdSense publishers’ earnings overall from the move to a first-price auction to be neutral. When Ad Manager moved to a first-price auction, there was a neutral to slightly positive impact to publisher earnings on average,” the company said.
The post Google AdSense moves to a first-price auction model appeared first on Search Engine Land.
Achieving sustained success in the mortgage brokerage industry requires a diverse marketing strategy. Unfortunately, this is often easier said than done. After all, digital marketing is a constantly evolving, complex field.
With this in mind, the team at BNTouch has outlined several digital marketing tips for mortgage brokers. These tactics can help you generate more leads, stay connected with clients, and grow your business. Plus, your loan officers will have an abundance of quality prospects to target with their marketing efforts!
Read on to find out how you can revitalize your digital marketing efforts in 2021.
Whether you are a relatively new loan officer or have been involved in the industry for years, the first step to digital marketing is to identify your ideal audience.
Even if you think you know who you should be targeting, reassess your current practices. As your organization grows, your ideal customer base may change as well.
A great way to ensure that you focus your energies on the right clients is to create “customer personas.”
These personas list out characteristics that you would like to find in a quality lead. You can also create personas for buyers who aren’t quite ready to purchase and low-quality leads. After you create these templates, your loan officers can easily sort through prospects and identify quality leads.
Guest posting is the practice of submitting blog content to other companies to publish it on their sites. Focus on companies related to the mortgage industry, including lenders, real estate agents, and title agencies. A great place to start is the companies that you already work with daily.
As you’re crafting these guest posts, make sure to include links to your page. Discuss the type and quantity of links with the host ahead of time to prevent unforeseen disagreements.
Links from another site to your page are known as backlinks. Google uses these types of links in the ranking process. Having high-quality backlinks can help your site reach page one of the search results.
As a mortgage broker, you have to make effective use of various types of media. While optimized blog content is a core part of any digital marketing strategy, it will not generate enough quality leads on its own. You must also incorporate videos and images into your digital marketing campaigns.
Videos are particularly appealing to modern consumers. You do not need to flood your website with videos or reach a million YouTube followers for this strategy to work. A few carefully placed pieces of media content go a long way.
For instance, placing a video on your “About Us” page makes your team seem more personable, which can draw in quality leads.
If you don’t have a website, creating one should be a top priority. Every mortgage broker needs their site if they hope to be competitive in the modern marketplace.
While you may still receive your fair share of phone calls, many consumers search for mortgage services online. If you don’t have a website, a high number of leads will immediately move on to the next loan officer — one who does show up online.
Even if you have a website, it is important to review it from time to time. You should access it from your smartphone to ensure that it is optimized for mobile, too. If it is not, reach out to a web development firm to remedy the issue immediately. A large percentage of consumers browse the web primarily from their smartphones.
If your website is not attractive and engaging or fails to work well on mobile devices, you’ll drastically limit your reach. Revitalizing your online presence starts and ends with a great site.
In addition to a website, social media pages are must-have marketing tools that every mortgage broker should be using.
Social media platforms like Facebook and Instagram offer a ton of free advertising opportunities. Once your platforms are live, begin to post quality content regularly. Your content might include success stories, links to recent blogs, and other materials that will encourage consumers to visit your site.
We recommend focusing your energy on a couple of social media platforms. Create accounts on the platforms that are most popular with your target audience. As previously mentioned, identifying your ideal audience is an essential first step to rejuvenating your digital marketing strategy!
In the mortgage industry, email marketing is an incredibly useful tactic. Sending past clients periodic emails to celebrate birthdays or notify them about refinancing opportunities can pay off over time.
These messages will keep your company at the forefront of customers’ minds when it comes time to buy another home or refinance. If possible, offer them the opportunity to work with the same loan officer, too.
Email marketing can also be used to turn prospects into qualified leads. When your team receives information about a prospective client via a contact card or phone call, add their emails to your list.
Sending relevant email communications that are concise and engaging can draw customers deeper into the sales funnel. This allows your loan officers to close the deal.
When you have a top-quality mortgage CRM, it’s easy to send out effective marketing via email. Add dates and tags to buyers so that you’ll never forget to celebrate the anniversary of their home purchase. Then, your system can send out custom messages when the dates arrive!
Retargeting is the practice of directing specific advertisements at users that have previously visited your site. This strategy is very effective because most clients do not select a mortgage broker after a single visit to their website!
Clients that have provided you with contact information clicked through multiple pages on your website, or interacted with your content are prime candidates for retargeting.
You can set up your retargeting campaign through Google Ads. If you aren’t familiar with the platform, the best mortgage CRMs often offer tools designed to make it easy to create new campaigns with popular platforms.
Consumers are wary about advertising content. However, they are much more receptive to information provided by other consumers. This concept holds true in the mortgage industry as well.
Encourage your clients to review your business online. The more positive reviews you have, the better.
Reach out to past clients and politely encourage them to leave you a review through a survey or Google. You can easily do this with a CRM that sends out automated requests about a month after a loan is closed.
Once you have accumulated several positive reviews, highlight this customer feedback on your website or share some success stories in an email marketing campaign.
You can also use this feedback to improve your customer service. These reviews can help you identify what your loan officers are doing well and what they need to be doing better.
The mortgage industry is founded on relationships. You have to work with buyers, realtors, title agencies, and other industry professionals to close a loan. Instead of simply using these relationships to close mortgages, leverage them to market your services.
Partner with experts in the mortgage industry. Devise a mutually beneficial agreement that will help you generate leads while also helping their businesses.
If you are a new loan officer, consider working with a more experienced broker. They may be able to help you develop your lead generation skills and teach you how to better serve your customers.
Much like generating organic traffic, building a social media following can be extremely time-consuming. If you want to reach a large audience fast, consider partnering with social media influencers. These individuals are social media personalities that have thousands or tens of thousands of followers.
An influencer can help to market your services on their social media page. This relationship gives you access to their massive audience. In exchange, influencers receive payment, free services, or both.
As you’re searching for an influencer, don’t focus entirely on the size of their following. Instead, make sure that the influencers you work with are a good fit for your mortgage brokerage.
Implementing these tactics can help you massively increase your lead generation capabilities. But you still need to track and distribute your leads effectively.
So what’s the best way to do it? Use an innovative mortgage CRM, like the BNTouch platform.
Our versatile software includes a full suite of tools. With it, you will be able to automatically distribute leads, track team performance, and analyze key metrics. You will also be able to automate marketing strategies so that you can focus on serving your clients.
BNTouch offers scalable solutions for mortgage brokerages of all sizes. Whether you are an independent loan officer or manage an entire mortgage enterprise, our platform can accommodate your needs.
Contact us today for a free demo and see our technology firsthand!
The post 10 Digital Marketing Tips for Mortgage Brokers and Loan Officers in 2021 appeared first on .
After 18 months of heartbreak and hardship, business owners are ready to bounce back.
Eight out of ten small businesses are fully recovered from 2020’s recession, and 94% are looking to partner with agencies to fuel long-term growth. Join growth expert Jen Spencer, chief revenue officer, SmartBug Media, who will share how SMBs position themselves for growth and where they need help, including tactics like paid search, SEO, website development, blogging and attaining customer reviews.
Register today for “Growth Strategies for SMBs and the Agencies That Serve Them,“ presented by CallRail.
The post Strategies to fuel your long-term growth appeared first on Search Engine Land.
Search Engine Land’s daily brief features daily insights, news, tips, and essential bits of wisdom for today’s search marketer. If you would like to read this before the rest of the internet does, sign up here to get it delivered to your inbox daily.
Good morning, Marketers, where does diversity start in marketing?
My friend Chima Mmeje, an SEO content writer and strategist for SaaS and tech, tweeted this week asking why there were no Black in-house marketers. “What’s the barrier to entry?” she asked.
We’ve written about the issue on Search Engine Land and MarTech. Here’s what we’ve found: When we talk about the career pipeline that leads to more diverse and inclusive leadership, it starts at the very beginning. The National Association of Colleges and Employers found that Black students are more likely to be in unpaid internships than their white peers – and that has a direct effect on diversity in the workforce.
“Many organizations use their internship programs to feed full-time hiring; consequently, looking at how they source and select interns is critical for those that are committed to diversifying their workforces,” said Shawn VanDerziel, NACE executive director. If you can’t afford to take an unpaid internship, then you’re already at a disadvantage in your career.
Not only is it harder for Black marketers to break into their careers of choice, but they are less likely to be promoted. Mercer, a human resources consulting firm, released data that showed only 12% of support and operations staff are Black (versus 64% white), and that number continually decreases as Black employees are not promoted to higher levels within companies. The Association of National Advertisers (ANA) in a 2020 report found that only 6% of its total members and 3% of the members who are CMOs are Black.
So what can we do? SearchPilot recently posted the results of their diversity and inclusion transparency report, which they commit to publishing each year. Dr. Lauren Tucker, founder of Do What Matters, told MarTech that inclusion comes first, “I call it ‘inclusion, equity and diversity.’ Diversity is the outcome of inclusion and equity.” Dr. Tucker says that while Chief Diversity Officers are great, they have to be funded and supported, and employee resources groups are often a great way to get started if your company isn’t big enough for a CDO.
Carolyn Lyden,
Director of Search Content
Beginning June 30, 2022, responsive search ads (RSAs) will be the only search ad type that can be created or edited in standard search campaigns, Microsoft Advertising announced in its October product updates on Tuesday.
The announcements also included reporting improvements for Microsoft Audience Ads with view-through conversions, In-market Audiences expansions, an open beta for auto-generated remarketing lists, several Microsoft Audience Network updates and expansions and more.
Why we care. The deprecation of ETAs aligns Microsoft Advertising with Google Ads, making it easier for advertisers to work across both platforms without having to account for any differences in available search ad types. As we stated when Google first made this announcement, this move means that advertisers will have less direct control over their accounts and will have to get accustomed to working with Google and Microsoft’s machine learning, if they haven’t already. We recommend testing your ETAs and RSAs to figure out what works best well before the June 30, 2022 deadline.
Read more details on the updates here.
In September, Eric Enge ran a poll on Search Engine Land to see if readers would like to have an instruction in robots.txt to mark pages for No Indexation. The results are in:
In total 84% of the 87 respondents said “yes,” they would like this feature. Some of the reasons offered for wanting this feature were:
Reasons for saying no included:
Diversity, equity, and inclusion are ongoing efforts and something we need to commit to every day. This is why we’re excited to announce the second annual Search Engine Land Award for Advancing Diversity and Inclusion in Search Marketing to celebrate those individuals or organizations who are affecting real change.
The deadline for nominations is October 8, 2021.
You may nominate as many organizations or individuals as you feel deserve the recognition. We ask that you highlight specific initiatives conducted by the nominee and that in the nomination form you include the contact of someone who can “second” that nomination.
Nominate a person or organization now.
With SMX Next coming up (check out the agenda and register here!), we wanted to recap some of the awesome coverage from our other events this year. Check out the how-tos, myths, and mistakes to avoid:
Bing accounts for 26.5% of all desktop searches in the U.S., according to Comscore (April 2021). With the recent prevalence of working from home, people are spending more time on their desktop computers, which might also mean that more people are now using Bing.
Bing’s ranking algorithm is dynamic. “The ranking algorithm is a gigantic machine learning model and it’s evolving constantly,” said Frédéric Dubut, principal project PM manager, core search & AI at Microsoft. Before you integrate the following advice into your SEO strategy, keep in mind that better rankings aren’t guaranteed just because you optimize for the specific factors highlighted in the Bing Webmaster Guidelines.
This potentially means that, if everyone starts prioritizing one ranking criteria, then that signal may become less indicative of relevance and Bing’s algorithm may assign less weight to it. Instead of cherry-picking ranking factors to optimize for, we recommend that you cover all the bases to the best of your ability while keeping in mind how Bing treats the following elements of search.
Quality and credibility. To determine the quality and credibility of a site, Bing looks at factors like the site’s reputation, the author’s reputation, transparency of authorship, completeness of content and the level of discourse.
In terms of site reputation, “It is really about mapping and understanding that this website is an authority for this specific domain,” Fabrice Canel, principal program manager at Bing, said during an edition of Live with Search Engine Land.
Want to keep reading? Check out the full guide here.
The post Microsoft Advertising will deprecate ETAs too; Wednesday’s daily brief appeared first on Search Engine Land.
Nothing brings people together like watching their national team compete on the world’s stage. The excitement unites a diverse set of fans, each with their own stories to be told.
In the martech world, a tool unites customer data just like fans at a national sporting event: a Customer Data Platform (CDP). A database often used by marketers and CX teams to ingest customer data from relevant platforms and channels, CDPs provide a single, unified view of your customer.
The U.S. Soccer Federation was looking for a solution to revamp its antiquated digital infrastructure, including an outdated website, lack of mobile app and plenty of tedious, manual tasks to keep track of customer data. Before investing in Treasure Data’s CDP solution, U.S. Soccer mapped out three business challenges:
To learn more about why U.S. Soccer decided to invest in Treasure Data’s CDP to address the above challenges and more, join Ross Moses, senior director of analytics and insights at U.S. Soccer Federation, and J.T. Wash, senior account executive at Treasure Data, in their informative MarTech session.
This exciting, data-rich case study highlights how U.S. Soccer improved its fan experience and increased member loyalty by creating a unified view of its customer journey.
After this session, you’ll be able to:
Register and attend the session here and learn how to drive relevant, real-time customer experiences throughout the entire customer journey with a CDP.
The post U.S. Soccer unifies the fan experience with Customer Data Platforms appeared first on Search Engine Land.
Search Engine Land’s daily brief features daily insights, news, tips, and essential bits of wisdom for today’s search marketer. If you would like to read this before the rest of the internet does, sign up here to get it delivered to your inbox daily.
Good morning, Marketers, and what is content?
I’ve been chatting with someone on the marketing ops side of marketing a few times a month just to get an idea of how they think the upcoming cookieless world will affect things on their end. She was telling me about tagging and attribution in the many CRMs that she helps clients with and the conversation drifted to what happens in “the middle.”
Her work encompasses the beginning (planning campaigns and tagging) and the end (collecting attribution data and making decisions based on it). But she didn’t have a lot of insight into the middle. Her assumption was that all SEO and PPC were essentially just “content.”
We walked through the technical basics of SEO and PPC and she asked, “So does anyone not need content?” And it got me thinking about the definition of content. We just assume it means pages or posts on our sites, but content can also encompass Google My Business and social media posts, ad creative and landing pages, infographics and videos, and more. Content is what makes the web work. It goes beyond both SEO and PPC but is also at the core of what they are.
P.S. Love my musings? Join me for more at my SMX Next keynote. If you don’t, then you should still join for other people’s expert knowledge. 🙂
Carolyn Lyden,
Director of Search Content
“Many of the tracking tools are showing levels that we have not seen since a Google core update. Moz reported 101 degree weather, Advanced Web Ranking was at the top of its chart,… and the other tools are also super hot,” said Barry Schwartz on Search Engine Roundtable.
The SEO community has only seen some chatter, however, but it was a big deal to some marketers: “USA traffic [is] still way off, down about 30% from the first two weeks of Sept and still dropping. USA has dropped so much that my UK traffic is now 2/3 of USA, for a population 6+X larger,” said one SEO. “Big movements across all topics and countries. The volatility is similar to a core update. It’s the update they started rolling out yesterday,” said another.
Why we care. If you saw big dips in USA traffic last week or over the weekend, it might be attributed to what’s going on algorithm-wise. Let your clients and stakeholders know what happened before they come to you asking about metrics anomalies.
Facebook and Instagram were down for over two hours yesterday. The outrage resulted in quite a few funny tweets, as Twitter was one of the only social media platforms still up at the time. The error message suggested a Domain Name System (DNS) error. Those who used Facebook as a login verification for other apps also could not log in to those third-party systems. The outage isn’t the first for Facebook. The app and websites were down in March and July this year, too.
Why we care. Make sure to mark the outage in your Analytics if you rely heavily on leads or traffic from social. The outage will have a heavy impact on both paid and organic social media campaigns, but fortunately only for a short period of one day.
Also read: What marketers can do next time a major social network (ahem, Facebook) goes down
Google is showing trending keyword data within Google keyword planner. “This is a small experiment. We’re always testing new ways to improve our experience for our advertisers and users, but don’t have anything specific to announce right now,” a Google Ads spokesperson told Search Engine Land. Hat tip to Arbab Usmani for sending us a screenshot.
Support for Responsive Video Ads in scripts. Reminder: Last week Google Ads launched support for responsive video ads in Google Ads scripts. If you were using the TrueView for action campaign type, you must update your code to use the new video ad type.
How to use content and PR to build deep links. Deep links can drastically improve organic performance in the long term, but they are hard to get and we can’t influence where publishers link to. Screaming Frog has a guide to work around those issues, though.
Google local search trends: Socialization. Google’s local platform — comprising Google My Business, Google Maps, and the local component of Google Search — has become, under our noses, a massive social network. Google has achieved this status not through traditional methods of connecting users to each other, but by allowing and encouraging users to share their experiences, questions, and opinions about local businesses in a variety of forms and at a massive scale.
Sure you may know what a pre-roll, mid-roll, and post-roll ad is in podcast advertising, but how about inventory source targeting or a baked-in ad? Podcasting is experiencing a serious boom, and with that comes all the different ways advertisers can reach those very targeted audiences of listeners.
SXM Media recently published a guide to podcast advertising lingo to clarify the new jargon that’s come along since podcasts have really taken off (especially in the ad world).
So a baked-in ad is “part of the actual podcast recording, meaning they live with that episode forever.” This is the opposite of a dynamically inserted ad which gives podcasters “the ability to insert ads, either client-provided, announcer-read, or host-read, at the time of download. This ensures that all ads are relevant to the time of year (we wouldn’t want any holiday ads in the peak of summer) and to the listener themselves.”
Oh, and inventory source targeting is “the ability to target the inventory of your favorite publishers at-scale, like Stitcher or Pandora.” Make sure to check out the guide to stay up to date on the latest podcabulary — there will be a pop quiz later.
The post Facebook: Have you tried turning it off and on again; Tuesday’s daily brief appeared first on Search Engine Land.
In September, I put up a poll here on Search Engine Land to see if readers would like to have an instruction in robots.txt to mark pages for No Indexation. Today I’ll present the results along with a review of what the key issues are (and why Google won’t be adding support for this).
In the current environment, robots.txt is used exclusively for guiding the behavior of web crawling. Further, the current approach to marking a page “NoIndex” is to place a tag on the page itself. Unfortunately, if you block it in robots.txt, Google will never see the tag and could still potentially index the page even though you don’t want that to happen.
On large sites, this presents some challenges when you have different classes of pages that you’d like to both block from crawling AND keep out of the Google index. One way that this can happen is in complex faceted navigation implementations where you have pages that you create that have significant value for users but end up presenting way too many pages to Google. For example, I looked at one shoe retailer website and found that they have over 70,000 different pages related to “Men’s Nike shoes.” This includes a wide variety of sizes, widths, colors, and more.
In some tests that I have participated in with sites with complex faceted navigation like the example I shared above we have found this large quantity of pages to be a significant problem. For one of those tests, we worked with a client to implement most of their faceted navigation in AJAX so the presence of most of their faceted nav pages was invisible to Google but still easily accessed by users. The page count for this site went from 200M pages to 200K pages – a thousand to one reduction. Over the next year, the traffic to the site tripled – an amazingly good result. However, traffic went DOWN initially, and it took about 4 months to get back to prior levels and then it climbed from there.
In another scenario, I saw a site implement a new e-commerce platform and their page count soared from around 5,000 pages to more than 1M. Their traffic plummeted and we were brought in to help them recover. The fix? To bring the indexable page count back down again to where it was before. Unfortunately, since this was done with tools like NoIndex and Canonical tags the speed of recovery was largely impacted by the time it took Google to revisit a significant number of pages on the site.
In both cases, results for the companies involved were driven by Google’s crawl budget and the time it took to get through enough crawling to fully understand the new structure of the site. Having an instruction in Robots.txt would rapidly speed these types of processes up.
I had the opportunity to discuss this with Patrick Stox, Product advisor & brand ambassador for Ahrefs, and his quick take was: “I just don’t think it will happen within robots.txt at least, maybe within another system like GSC. Google was clear they want robots.txt for crawl control only. The biggest downside will probably be all the people who accidentally take their entire site out of the index.”
And of course, this issue of the entire site (or key parts of a site) being taken out of the index is the big problem with it. Across the entire scope of the web, we don’t have to question whether this will happen or not — it WILL. Sadly, it’s likely to happen with some important sites, and unfortunately, it will probably happen a lot.
In my experience across 20 years of SEO, I’ve found that a misunderstanding of how to use various SEO tags is rampant. For example, back in the day when Google Authorship was a thing and we had rel=author tags, I did a study of how well sites implemented them and found that 72% of sites had used the tags incorrectly. That included some really well-known sites in our industry!
In my discussion with Stox, he further noted: “Thinking of more downsides, they have to figure out how to treat it when a robots.txt file isn’t available temporarily. Do they suddenly start indexing pages that were marked noindex before?”
I did also reach out to Google for comment, and I was pointed to their blog post when they dropped support for noindex in robots.txt back in 2014. Here is what the post said about the matter:
“While open-sourcing our parser library, we analyzed the usage of robots.txt rules. In particular, we focused on rules unsupported by the internet draft, such as crawl-delay, nofollow, and noindex. Since these rules were never documented by Google, naturally, their usage in relation to Googlebot is very low. Digging further, we saw their usage was contradicted by other rules in all but 0.001% of all robots.txt files on the internet. These mistakes hurt websites’ presence in Google’s search results in ways we don’t think webmasters intended.“
* Bolding of the last sentence by me was done for emphasis.
I think that this is the driving factor here. Google acts to protect the quality of its index and what may seem like a good idea can have many unintended consequences. Personally, I’d love to have the ability to mark pages for both NoCrawl and NoIndex in a clear and easy way, but the truth of the matter is that I don’t think that it’s going to happen.
First, I’d like to acknowledge a flaw in the survey in that question 2, a required question, assumed that you answered question 1 with a “yes”. Thankfully, most people who did answer “no” on question 1 clicked on “Other” for question 2 and then entered in a reason for why they didn’t want this capability. One of those responses noted this flaw and said, “Your poll is misleading.” My apologies for the flaw there.
The overall results were as follows:
In total 84% of the 87 respondents said “yes,” they would like this feature. Some of the reasons offered for wanting this feature were:
Other reasons for saying no included:
There you have it. Most people who responded to this poll are in favor of adding this feature. However, bear in mind that the readership for SEL consists of a highly knowledgeable audience – with far more understanding and expertise than the average webmaster. In addition, even among the yes responses received in the poll, there were some responses to question 4 (“would this feature benefit you as an SEO? If so, how”) that indicated a misunderstanding of the way the current system works.
Ultimately though, while I’d personally love to have this feature it’s highly unlikely to happen.
The post We asked, you answered: Should Google add noindex support to robots.txt? Poll results appeared first on Search Engine Land.