How to Optimize Your Website Design with the User in Mind


Too often, the aesthetics of a website are given more thought than how a user will navigate it. However pretty a site may be, it is useless if users can’t find what they’re looking for. Understanding how your users are finding, interacting with, and purchasing on your site will allow you to design a more user-friendly experience, eliminate unnecessary or confusing calls to action, and ultimately increase conversions.

The first element of designing a successful site is understanding user intent.

What Is User Intent?

User intent is the information the user is after when conducting a search query. It’s usually thought of in the context of keywords and search queries, but for a successful user experience, it is important to expand this definition.

Let’s go beyond just thinking in terms of SEO and start thinking of user intent as the users’ goals on your site. When you take this into account, you can go past just getting customers to your site. You are now thinking about the user’s entire experience on your website and ultimately how to convert them into customers.

When studying user intent, there are three categories that can help you organize the traffic patterns you are seeing. Once you have identified each of these groups, you can address their individual needs to make sure their experience on your site goes as you intended and, more importantly, as they expected.

First are the informational users. These are the users that are looking for more information about your product, your company, or how to contact you.

Next, we have the educational users. Educational users are looking to learn about industry-specific topics. They are trying to find answers on how to solve a particular problem or fulfill a need, but probably don’t have a specific solution in mind yet.

Finally, our favorite group, the transactional users. Transactional users are ready to download something, sign up for something, or make a purchase. These users come to our site with a clear plan of action, and so it is of the utmost importance we make it easy for them to execute that intent.

The first element of designing a successful site is understanding user intent.
Click To Tweet

How Do You Determine User Intent?

Now that you have categorized different types of users, you can start identifying and analyzing each group’s traffic pattern. A great way of determining your users’ intent is by analyzing the Users Flow report from Google Analytics. With this report, you can track where the users started (the source), what they did while on your site (the traffic), and where they left your site (hopefully, the “Thank you for your purchase” page).

Learning how to set up and configure the users flow report is well worth your time and effort. Once you have the report up and going, you can begin to analyze the flow of your users and get a better understanding of how you can improve your site’s design.

Once you have an understanding of how these three types of users are experiencing your site, you can begin to identify and resolve any trouble spots. The best place to focus your efforts is your navigation bar.

Organizing Your Nav Bar

By focusing your efforts on your nav bar, you will be able to address any navigation issues you identified in your Users Flow report. First, identify categories and subcategories for your company, products, and services that are clearly defined and easily understood by your users. This is creating your content hierarchy. Think of this like organizing the aisles of a grocery store or creating an index for the content of a book.

This isn’t the place where you want to reinvent the wheel. Users expect some consistency with established convention. Stray too far from these conventions, and users can get frustrated and quickly leave your site.

When deciding how to best organize your navigation bar and which categories to include at this top level, there are a couple of different approaches to consider:

User Intent Buckets

One tactic you can use is organizing by the different user categories. For our informational users, we need to create an “About Us” page. This is where we can answer questions about our company, our products and services, and the area or clients we serve.

For our educational users, we need a blog or some other form of instructional content. This is where we can provide content that highlights trends in the industry, discusses issues, and clarifies common industry questions.

Finally, for our transactional users, we need to provide easy access to product-specific pages that make the purchasing process clear and simple.

For example, if you take a look the nav bar at the top of the Convince & Convert site, you will see:

  1. Informational users can get everything they need about the company from the categories in the top right.
  2. Educational users will navigate to the far right to get to the blog or podcasts.
  3. Transactional users will use the categories on the left to start the purchase process.

Convince and Convert nav bar

Types of Customer

If your target audience can be grouped into different categories with varying needs, then this type of organization will work best for you. Organizing your nav bar based on the type of customer allows you to group specific products and services or content together that would be most beneficial for that customer type. This saves your customer time by putting all the information they would need in one place.

It’s important to note that if your customer base cannot be clearly separated into types of customers, this type of navigation can create confusion and frustration. You want to be sure your customer categories are very distinct.

Let’s look at the nav bar for TransUnion’s SmartMove division, for example. The customer base can be clearly split into (1) landlords and (2) renters, so they have set up their nav bar to clearly reflect this. The subcategories included in each category are specific to that type of customer, making it easier for customers to find what they’re looking for.

TransUnion nav bar



A product-centric approach takes a look at what clearly definable products or services are provided and builds the navigational categories from there. If you are able to easily categorize your products or services, this can help your user gain more information about a category of products before navigating to individual products within the same category.

A good example of this approach is from the health and beauty company Colorescience. By organizing their products into clearly defined categories, a user can easily find the product they are looking for within the category that it would fall under. For example, if a customer is looking to learn more about products offering (1) UV protection, they know exactly where to go.

Colorescience nav bar


Now that you have an idea of what organizational approach you want to take with your navigation bar, it is important to avoid some common mistakes.

Nav Bar Dos and Don’ts

Limit the Number of Categories

The main goal of the nav bar is to provide clear choices that enable users to make quick decisions on how to accomplish their intended goals. Providing too many categories makes it harder for the user to know which path to take.

Limit your navigation bar to around five to seven categories. Not only does this make it easier for your user to know exactly where to go, but it also makes your site more visually appealing across multiple devices and screen sizes.

To see a good example of what not to do, we’ll need to go back a couple years to Microsoft’s site from 1999. Back then, they had three separate nav bars that each contained too many categories. Users presumably had a very difficult time finding their way around the site, since they were given so many options.

Keep Wording Clear

Choose specific words and phrases, and try to limit them to 12 characters or less. Again, you want to stick to convention here. For example, “About Us” is commonly used for company and product information. It is easily recognizable, and your users will quickly identify the purpose of this link. Changing it up to “Who We Are” can slow this process down and frustrate or confuse some users.

You also want keep wording short and sweet. Instead of using “Pricing Information,” just use “Pricing.” Instead of “Our Weekly Blog,” just stick with “Blog.” The simpler, the better!

Let’s take a look at healthcare company Inovalon and their website. While it isn’t necessarily wrong to word their categories as they do, it may make it difficult for users to determine which category to click on, especially considering they do not implement sub-menus. For example, if a user is looking for more specifics about what the company does, they may be able to find this information in each of the four categories.

Inovalon nav bar

Actions on the Right

Use second nature to your advantage. Because we read from left to right, users naturally expect informational categories to be listed on the left, as they want to learn more before taking a next step. Using that same logic, users expect to categories that allow them to take action, like “Contact Us,” to be on the right side, or the end of the nav bar. This allows a user to gather information and then take action in the order that they expect.

One exception is your home link. You want to keep the home link furthest to the left because users choose that link to return to the beginning, or make a backward action. Think of the rewind and fast forward buttons on a remote. When we want to move forward, we push the arrows pointing to the right, and when we want to go back to something, we push the arrows pointing to the left. It’s simply second nature.

While a horizontal navigation bar is most common, there’s nothing wrong with a vertical nav bar. The right-to-left rule obviously doesn’t apply for vertical nav bars, but a similar top-to-bottom rule takes its place. Just like in the right-to-left rule, users read top-to-bottom, so it’s only natural to organize your nav bar in that way.

Our final example comes from Zervice, a service that syncs up with Salesforce. They make use of a vertical nav bar, but automatically start the user at the bottom of the page. While they do make it clear where users should start (1), it doesn’t allow for a logical navigation through the website.

Understanding the different types of users that visit your page and their intent once on the page can help significantly improve the flow of your website. You now know how to use user intent and traffic patterns to better understand how your users are navigating your site. You can use this information to improve the organization of your site and allow for a better user experience. By optimizing your site’s design with the user in mind, you will ultimately make it easier for customers to make a purchase.

Go ahead! Start optimizing!

Get a weekly dose of the trends and insights you need to keep you ON top, from the strategy team at Convince & Convert. Sign up for the Convince & Convert ON email newsletter.


The world is flat, if we let businesses keep it that way


By Mark Schaefer

Every few years, I find a business book that upends my way of thinking. One of those events occurred in 2005 when I picked up The World is Flat by economist and columnist Thomas Friedman.

The book envisioned a world where businesses — not governments or politicians — would help create a safer and more predictable world.

Friedman pointed to the growing online connectivity of societies, the globalization of commerce, and the economic interdependence of nations as stabilizing trends. He forecast an inevitable end to many national tensions because of the business implications. For example, why would China ever pick a fight with the U.S. — They had too much to risk by jeopardizing a relationship with their most important trade partner. Increasingly, business would be the glue that held the world together.

Since this book was written more than a decade ago, much of what Friedman predicted has come true, but unfortunately we still have politics messing things up. I believe he’s correct in that eventually business will help keep peace in the world and I saw this first-hand last week.

Business as the glue for world peace

On my recent business trip, I had a chance to visit with people from different countries who have had historical conflicts with the U.S., and with each other.

Perhaps the highlight of my 10-day trip to Poland, Russia and The Czech Republic was a workshop with marketers from the Carlsberg Brewing Group in the lovely city of Saint Petersburg.

It’s funny how many warnings and jokes were made to me by my friends before my Russia trip — certainly the relationship between the U.S. and Russia is strained and complicated. Some political observers have characterized the connection between the countries as the worst in decades.

And yet throughout my stay, I was treated with nothing but respect and kindness. Like marketing geeks everywhere, we enjoyed swapping stories of business successes and challenges. We were united by our love of innovation and our drive for progress.

Everywhere I went in Eastern Europe I saw people enjoying American movies, music, fashion, and food (Starbucks is ubiquitous!). It didn’t look like a politically tense situation based on the “votes” of commerce and everyday people just trying to get along.

Likewise, Russia and the Ukraine have been actively at war since 2014 and Russia has also had escalating political tensions with Belarus and other countries in the region. But as business leaders from these different nations gathered in one setting, they met together in peace, and as friends. We all care about our companies, we all want to achieve prosperity, we all love our families.

I don’t want to minimize the complexity of our world. But at the same time, this was a great reminder to me of the unifying force of business. The politicians will be politicians. Much of their success seems to be based on driving divisiveness and fear. But in business, the world is flat. Progress has to be made through collaboration, understanding, and generating “win-wins.” That seems like a much healthier plan.

My trip was beautifully Friedman-esque as I witnessed people whose countries might be national “rivals” joke with each other, embrace each other as friends, and even share a beer together. We should be so lucky to see our national leaders doing the same thing.

SXSW 2016 3Mark Schaefer is the chief blogger for this site, executive director of Schaefer Marketing Solutions, and the author of several best-selling digital marketing books. He is an acclaimed keynote speaker, college educator, and business consultant.  The Marketing Companion podcast is among the top business podcasts in the world.  Contact Mark to have him speak to your company event or conference soon.

Book link is an affiliate link.

The post The world is flat, if we let businesses keep it that way appeared first on Schaefer Marketing Solutions: We Help Businesses {grow}.


The 3 Behaviors Driving the Most Creative Content Marketers


Editor’s note: You may have missed this article when CMI published it last year. We’re sharing it now because it’s the time of the year when we like to talk cold sweets (and remind you about the qualities needed for creative content marketers.)

Before we get into all that delicious content marketing stuff, let’s talk about eating ice cream for a second. (Stay with me, folks. It’s gonna get weirder before it gets normal again.)

When you eat a bowl of ice cream, is your goal to get to an end result as fast as possible? Do you turn to a friend or maybe a professional ice-cream-eating freelancer and say, “Hey, can you finish this bowl of ice cream for me? I just really want a messy bowl.”

That’d be insane, right? The best part of eating ice cream is the process of eating the ice cream. And since we’re so infatuated with the process itself, some interesting behaviors unfold – namely, we tinker. We make the ice cream better. We add toppings. We put it into things, onto things, and next to things.

Because of our focus on the process – not our obsession with the end itself – we innovate.

What does this have to do with content marketing? Nothing. I just wanted to talk about my favorite dessert.

Kidding – of course the answer is “everything!”

If you study the most creative content marketers, it turns out that they approach their work much like most of us approach a bowl of ice cream: They make the process the point, not the end results. And as a result, they get better end results.

Too much #contentmarketing goes through the motions. The best find joy in the process, says @JayAcunzo.
Click To Tweet

There’s a psychological benefit to that behavior too. (That’s right – science is at play here, not just one man’s ramblings on frozen treats.)

And that science is just one of three key behaviors driving the most creative among us.

1. Truly creative content marketers make work intrinsic, not ‘telic’

Here’s the science behind the ice cream metaphor: When you eat ice cream, you’re intrinsically motivated to eat it. You do it for its own sake regardless of the end result.

The opposite of something intrinsic is “telic.” When an action is telic, it is done for the final results – created for a definite end. When you focus too much on that end result, i.e., when an activity is entirely telic, it becomes something nobody wants to complete and few people do with gusto: a chore.

Here’s the rub, my friend: Marketers have turned content marketing into a telic activity. We want the formula. We want the best practice. We want to skip to the end result as quickly, cheaply, and repeatably as humanly possible.

Another way of saying this: Too much content marketing goes through the motions. But the very best among us find joy in the process. They LOVE creating the stuff, tinkering on their framework for doing so, and testing their process and workflow — just because, just for enjoyment. And this actually yields better end results.

Example: Julie Kim, director of content marketing at Slack

Slack is now the fastest-growing business app EVER. Its internal communication and chat tool has reached near ubiquity, especially among tech-savvy companies. And its content focuses on the content itself, not the results generated from it – and Slack gets better results in the end.

It all started with a focus on tone of voice. Slack is hell-bent on being “clear, concise, and human.” Most organizations in their shoes would focus on the leads or subscribers they wish to generate, or perhaps a self-aggrandizing statement like, “Our aim is to be the industry-leading source for jargon in the jargon-jargon space.”

In other words, Slack knows the right order for this work we do to succeed: Content. Then marketing. (Seriously, it’s called “content marketing,” not “marketing of content.” Sometimes I think we all need a sign in the morning over our beds: “Pants first. Then shoes.”)

One shining example of Slack’s amazing tone of voice is its podcast, Work in Progress, created in partnership with the branded podcast agency Pacific Content. Work in Progress is so good it’s syndicated to satellite radio. Listeners spend 20 or more minutes a week with Slack’s stories. (Remind me again how much we spend to get a few seconds from people’s day with most ads?)

Top to bottom, Slack’s organization knows why they do content: to produce great content. To be clear, concise, and human. To make work meaningful for others. Those are the first principles, the fundamental truths, behind the results they wish to deliver. Syndication to radio and millions of downloads for the show are just signals of success – signs that they’re doing great, INTRINSICALLY motivated work, not telic.

As Apple CEO Tim Cook once said, “We aren’t focused on the numbers. We’re focused on the things that produce the numbers.”

We aren’t focused on the numbers. We’re focused on the things that produce the numbers via @tim_cook.
Click To Tweet

Slack is. Are you?

2. Truly creative content marketers view the “quality or quantity” debate as a false, misleading choice – a debate not worth having

At Content Marketing World in 2015, I remember asking about 10 people before my speech whether they preferred to produce high-volume work or high-quality work. Everyone said, “High-quality, of course!”

But one intrepid content marketer named Colin paused. He looked at me and replied simply, “Why not both?”

I swear I almost hugged him. (I’m Italian. That’s my default greeting for other humans …)

I want you to imagine that one content producermarketer or otherwisewho WOWs you with his or her ability to make a TON of things, all incredibly well. Aren’t you downright jealous?

For us to even have a chance at being that good, we need to start in a much different place than asking, “quality or quantity?” As soon as we see those two things as juxtaposing ideas – or, worse, a choice we actually make – we’ve lost.

To the rest of the marketing world, on behalf of us create-first content marketers, allow me to clarify something: Our audiences want a lot of quality content. They don’t want a few things done well. They want EVERYTHING done well, all the time.

Audiences crave things they love. And when they get them, they want more. And more. And more.

Audiences crave things they love. And when they get them, they want more. @JayAcunzo
Click To Tweet

I asked a journalist friend once, “Do you write for quality or quantity?” He laughed in my face. “Both,” he said, “or I’m fired.” And the reason he can say that isn’t because he has some kind of superpower. It’s because he has a plan. He knows how to write an article. He knows terminology like lede, hook, and out. As a podcaster, I’ve learned about cold opens and episode rundowns.

In other words, the path toward doing more work at higher quality is the framework of HOW you do your work. You don’t need more budget, time, or team. You need a plan. In the same way that you can describe marketing to someone in terms of funnels and traffic patterns you should be able to describe HOW you write your blog posts. You should be able to teach HOW you create that podcast episode. When structuring your 800 words or 25 minutes, what are the component parts?

You need “content IP” where the “IP” does not stand for “I Produce.” If you can’t teach the creation part, you can’t scale without dumping increasing levels of crap onto the world.

Example: Andrew Davis, keynote speaker

Andrew is one of the most prolific speakers in marketing. Every year, he speaks everywhere from San Francisco to Sweden, to audiences ranging from fire chiefs to chief marketing officers. Andrew is a guy who absolutely has to deliver high-quality speeches and stories every single time he speaks, and he is speaking more and more times each year.

Quality. Quantity. Colliding. Feeling sick yet? Not our friend Mr. Davis, because he has some IP behind his speeches.

Andrew uses something called a “donut,” a term he pulled from his days producing television. A donut, as Andrew once explained to me, is a missing piece of content surrounded by repeatable or predictable content. Drawn as a circle, it resembles a donuta spot you must fill to fill in that circle.

If you’re delivering a speech, you might know the major problem you speak about, but you might have a hole for an illustrative story that you need to customize depending on your audience. After all, the same story that resonates with a group of content marketers might fall flat when presented to HR managers. Your story represents a donut.

Throughout Andrew’s 45-minute keynote talks, he has a handful of these donut holes, which he identifies for a given talk. He can curate stories or interview subjects in various industries. These stories, too, have a series of “beats” (yet another TV termthe moments that make up a good story, or the story style you’d like to tell). A “beat” might be something like, “Introduce me to the person by name and profession,” or, “Show me or tell me where they work and live.” These are the component parts to the story in the way that donut holes are component parts to the overall speech, which contains stories and other moments, like teachable lessons, data, and questions.

With every speech he delivers, Andrew fills in his donut holes with relevant stories.

With every story he researches, Andrew documents the appropriate beats to tell an Andrew-style story.

From a handful of speeches a few years ago, to 50-plus keynote talks last year, plus books, podcasts, blog posts, videos, and moreAndrew is scaling his output like crazy, maintaining quality all the while.

Your problem isn’t the tension between quality and quantityit’s the lack of preparation ahead of time so you never even feel it.

Your problem isn’t the tension between quality & quantity. It’s lack of preparation, says @JayAcunzo.
Click To Tweet

3. Truly creative content marketers are voracious consumers of their own material

We talk about “empathy” a lot in marketing. Empathy is the ability to understand and share the feelings of others. (This is not the same as sympathy, which we also need to feelbut only when our audience encounters our competitors! HEYOOO!)

Ahem. As I was saying: empathy! We need it. But it’s not like we’ve taken empathy courses. Nor is our boss sending us an analytics report in a panic because the data shows we’re not empathetic enough.

So how do we actually execute this idea of empathy? How can we understand what our audience will feel and therefore improve our ability to trigger a response?

We need to be our own biggest fans.

Now, a quick disclaimer: I don’t mean you should go home and pin your content marketing to your fridge, although that’s a nice little treat for the kids to stop daydreaming about being a firefighter and set their sights on our noble profession instead.

No, I mean simply this: We have to constantly consume our own work. Not edit it. Not audit it. Consume it.

As a senior in college, I used to mentor younger students who were also English majors. My favorite trick to make others better writers was to ask that they read a section out loud, either to me or softly to themselves. Instantly, you begin to uncover all the flaws or the areas you’d like to improve.

Sure, you’re not the customer. But that shouldn’t stop you from viewing your content through their eyes.

Example: Tim Urban, creator of Wait But Why

Tim Urban is a great writer. He’s built a blog audience of millions by publishing witty, stick-figure-ridden articles about complex topics like the unhappiness caused by using Facebook or the tricky concept of what “you” are. (Your brain? Your body? What IS the self?) Tim has given a TED talk about procrastination and was asked by Elon Musk to write about topics like colonizing Mars or the memory functionality of the human brain.

Tim is an amazing thinker and content creator, that much is clear. And while it’s unclear whether he consumes his own work, my guess is that he eats it up. I think he picks over the bones of an article like a hyena on a gazelle carcass, sucking and gnawing and clawing at every little scrap of the idea. How else would he use the tiniest of details to trigger the biggest of emotions in his readers?

For instance, rather than say, “We’re about to enter a period of rapid technological advancement,” he might draw this:


Additionally, if Tim is trying to make you feel something or react a certain way, he uses subtle details in his drawing to trigger that reaction, which he can do because, again, he’s seeing his work through your eyes. For example, when he writes about why people procrastinate, he introduces the concept of the rational decision-maker in your brain and the procrastination monkey like this:


Note the person first. He appears self-assured and reasonable, smiling and staring straight ahead. The copy reinforces this simple-yet-confident persona.

But then there’s the monkey. He’s saying something negative (“Nope!”), but Tim draws him with a big smile and raised arms. Those little effects ensure that the joke lands. The monkey is positively giddy in telling your brain, “Get something done today? No chance!” In a small drawing with little copy, you instantly get the tone of this little creaturehe’s troublesome and he relishes that fact.

Tim’s blog is read by millions, and yet he’s known for publishing less than once per week. And the secret behind it all is Tim’s ability to empathize with his audience and what they’d react to, from initial topic all the way through the tiniest detail of his writing and cartooning.

Want to wield empathy like a weapon? Want your audience responding with passion to your work? Don’t just ship your stuff into the abyss. Consume your own work. Act as your biggest critic. Be your biggest fan.

Content first. Then marketing.

As content marketers, we complain about the noise. But noise is not your problem. Sameness is. And while creativity can free you from making more commodity junk, you misinterpret what it means to be creative. So here’s the truth:

Creating something great doesn’t require a moment of genius. It demands a thoughtful, repeatable process.

Creating something great demands a thoughtful, repeatable process, says @JayAcunzo. #contentmarketing
Click To Tweet

Standing out isn’t about being bigger, better, or louder. It’s about being different.

And creativity isn’t a gift you’re given. It’s a work ethic.

So get to work.

As the highest-rated breakout speaker at Content Marketing World 2016, Jay has earned a spot on the main stage and will present a keynote  presentation at Content Marketing World 2017. Make plans today to hear him at CMWorld, September 5-8, in Cleveland, Ohio. Use the code BLOG100 to save an additional $100.

Cover image by Joseph Kalinowski/Content Marketing Institute

The post The 3 Behaviors Driving the Most Creative Content Marketers appeared first on Content Marketing Institute.


Evidence of the Surprising State of JavaScript Indexing


Posted by willcritchlow

Back when I started in this industry, it was standard advice to tell our clients that the search engines couldn’t execute JavaScript (JS), and anything that relied on JS would be effectively invisible and never appear in the index. Over the years, that has changed gradually, from early work-arounds (such as the horrible escaped fragment approach my colleague Rob wrote about back in 2010) to the actual execution of JS in the indexing pipeline that we see today, at least at Google.

In this article, I want to explore some things we’ve seen about JS indexing behavior in the wild and in controlled tests and share some tentative conclusions I’ve drawn about how it must be working.

A brief introduction to JS indexing

At its most basic, the idea behind JavaScript-enabled indexing is to get closer to the search engine seeing the page as the user sees it. Most users browse with JavaScript enabled, and many sites either fail without it or are severely limited. While traditional indexing considers just the raw HTML source received from the server, users typically see a page rendered based on the DOM (Document Object Model) which can be modified by JavaScript running in their web browser. JS-enabled indexing considers all content in the rendered DOM, not just that which appears in the raw HTML.

There are some complexities even in this basic definition (answers in brackets as I understand them):

  • What about JavaScript that requests additional content from the server? (This will generally be included, subject to timeout limits)
  • What about JavaScript that executes some time after the page loads? (This will generally only be indexed up to some time limit, possibly in the region of 5 seconds)
  • What about JavaScript that executes on some user interaction such as scrolling or clicking? (This will generally not be included)
  • What about JavaScript in external files rather than in-line? (This will generally be included, as long as those external files are not blocked from the robot — though see the caveat in experiments below)

For more on the technical details, I recommend my ex-colleague Justin’s writing on the subject.

A high-level overview of my view of JavaScript best practices

Despite the incredible work-arounds of the past (which always seemed like more effort than graceful degradation to me) the “right” answer has existed since at least 2012, with the introduction of PushState. Rob wrote about this one, too. Back then, however, it was pretty clunky and manual and it required a concerted effort to ensure both that the URL was updated in the user’s browser for each view that should be considered a “page,” that the server could return full HTML for those pages in response to new requests for each URL, and that the back button was handled correctly by your JavaScript.

Along the way, in my opinion, too many sites got distracted by a separate prerendering step. This is an approach that does the equivalent of running a headless browser to generate static HTML pages that include any changes made by JavaScript on page load, then serving those snapshots instead of the JS-reliant page in response to requests from bots. It typically treats bots differently, in a way that Google tolerates, as long as the snapshots do represent the user experience. In my opinion, this approach is a poor compromise that’s too susceptible to silent failures and falling out of date. We’ve seen a bunch of sites suffer traffic drops due to serving Googlebot broken experiences that were not immediately detected because no regular users saw the prerendered pages.

These days, if you need or want JS-enhanced functionality, more of the top frameworks have the ability to work the way Rob described in 2012, which is now called isomorphic (roughly meaning “the same”).

Isomorphic JavaScript serves HTML that corresponds to the rendered DOM for each URL, and updates the URL for each “view” that should exist as a separate page as the content is updated via JS. With this implementation, there is actually no need to render the page to index basic content, as it’s served in response to any fresh request.

I was fascinated by this piece of research published recently — you should go and read the whole study. In particular, you should watch this video (recommended in the post) in which the speaker — who is an Angular developer and evangelist — emphasizes the need for an isomorphic approach:

Resources for auditing JavaScript

If you work in SEO, you will increasingly find yourself called upon to figure out whether a particular implementation is correct (hopefully on a staging/development server before it’s deployed live, but who are we kidding? You’ll be doing this live, too).

To do that, here are some resources I’ve found useful:

Some surprising/interesting results

There are likely to be timeouts on JavaScript execution

I already linked above to the ScreamingFrog post that mentions experiments they have done to measure the timeout Google uses to determine when to stop executing JavaScript (they found a limit of around 5 seconds).

It may be more complicated than that, however. This segment of a thread is interesting. It’s from a Hacker News user who goes by the username KMag and who claims to have worked at Google on the JS execution part of the indexing pipeline from 2006–2010. It’s in relation to another user speculating that Google would not care about content loaded “async” (i.e. asynchronously — in other words, loaded as part of new HTTP requests that are triggered in the background while assets continue to download):

“Actually, we did care about this content. I’m not at liberty to explain the details, but we did execute setTimeouts up to some time limit.

If they’re smart, they actually make the exact timeout a function of a HMAC of the loaded source, to make it very difficult to experiment around, find the exact limits, and fool the indexing system. Back in 2010, it was still a fixed time limit.”

What that means is that although it was initially a fixed timeout, he’s speculating (or possibly sharing without directly doing so) that timeouts are programmatically determined (presumably based on page importance and JavaScript reliance) and that they may be tied to the exact source code (the reference to “HMAC” is to do with a technical mechanism for spotting if the page has changed).

It matters how your JS is executed

I referenced this recent study earlier. In it, the author found:

Inline vs. External vs. Bundled JavaScript makes a huge difference for Googlebot

The charts at the end show the extent to which popular JavaScript frameworks perform differently depending on how they’re called, with a range of performance from passing every test to failing almost every test. For example here’s the chart for Angular:

It’s definitely worth reading the whole thing and reviewing the performance of the different frameworks. There’s more evidence of Google saving computing resources in some areas, as well as surprising results between different frameworks.

CRO tests are getting indexed

When we first started seeing JavaScript-based split-testing platforms designed for testing changes aimed at improving conversion rate (CRO = conversion rate optimization), their inline changes to individual pages were invisible to the search engines. As Google in particular has moved up the JavaScript competency ladder through executing simple inline JS to more complex JS in external files, we are now seeing some CRO-platform-created changes being indexed. A simplified version of what’s happening is:

  • For users:

    • CRO platforms typically take a visitor to a page, check for the existence of a cookie, and if there isn’t one, randomly assign the visitor to group A or group B
    • Based on either the cookie value or the new assignment, the user is either served the page unchanged, or sees a version that is modified in their browser by JavaScript loaded from the CRO platform’s CDN (content delivery network)
    • A cookie is then set to make sure that the user sees the same version if they revisit that page later
  • For Googlebot:
    • The reliance on external JavaScript used to prevent both the bucketing and the inline changes from being indexed
    • With external JavaScript now being loaded, and with many of these inline changes being made using standard libraries (such as JQuery), Google is able to index the variant and hence we see CRO experiments sometimes being indexed

I might have expected the platforms to block their JS with robots.txt, but at least the main platforms I’ve looked at don’t do that. With Google being sympathetic towards testing, however, this shouldn’t be a major problem — just something to be aware of as you build out your user-facing CRO tests. All the more reason for your UX and SEO teams to work closely together and communicate well.

Split tests show SEO improvements from removing a reliance on JS

Although we would like to do a lot more to test the actual real-world impact of relying on JavaScript, we do have some early results. At the end of last week I published a post outlining the uplift we saw from removing a site’s reliance on JS to display content and links on category pages.


A simple test that removed the need for JavaScript on 50% of pages showed a >6% uplift in organic traffic — worth thousands of extra sessions a month. While we haven’t proven that JavaScript is always bad, nor understood the exact mechanism at work here, we have opened up a new avenue for exploration, and at least shown that it’s not a settled matter. To my mind, it highlights the importance of testing. It’s obviously our belief in the importance of SEO split-testing that led to us investing so much in the development of the ODN platform over the last 18 months or so.

Conclusion: How JavaScript indexing might work from a systems perspective

Based on all of the information we can piece together from the external behavior of the search results, public comments from Googlers, tests and experiments, and first principles, here’s how I think JavaScript indexing is working at Google at the moment: I think there is a separate queue for JS-enabled rendering, because the computational cost of trying to run JavaScript over the entire web is unnecessary given the lack of a need for it on many, many pages. In detail, I think:

  • Googlebot crawls and caches HTML and core resources regularly
  • Heuristics (and probably machine learning) are used to prioritize JavaScript rendering for each page:
    • Some pages are indexed with no JS execution. There are many pages that can probably be easily identified as not needing rendering, and others which are such a low priority that it isn’t worth the computing resources.
    • Some pages get immediate rendering – or possibly immediate basic/regular indexing, along with high-priority rendering. This would enable the immediate indexation of pages in news results or other QDF results, but also allow pages that rely heavily on JS to get updated indexation when the rendering completes.
    • Many pages are rendered async in a separate process/queue from both crawling and regular indexing, thereby adding the page to the index for new words and phrases found only in the JS-rendered version when rendering completes, in addition to the words and phrases found in the unrendered version indexed initially.
  • The JS rendering also, in addition to adding pages to the index:
    • May make modifications to the link graph
    • May add new URLs to the discovery/crawling queue for Googlebot

The idea of JavaScript rendering as a distinct and separate part of the indexing pipeline is backed up by this quote from KMag, who I mentioned previously for his contributions to this HN thread (direct link) [emphasis mine]:

“I was working on the lightweight high-performance JavaScript interpretation system that sandboxed pretty much just a JS engine and a DOM implementation that we could run on every web page on the index. Most of my work was trying to improve the fidelity of the system. My code analyzed every web page in the index.

Towards the end of my time there, there was someone in Mountain View working on a heavier, higher-fidelity system that sandboxed much more of a browser, and they were trying to improve performance so they could use it on a higher percentage of the index.”

This was the situation in 2010. It seems likely that they have moved a long way towards the headless browser in all cases, but I’m skeptical about whether it would be worth their while to render every page they crawl with JavaScript given the expense of doing so and the fact that a large percentage of pages do not change substantially when you do.

My best guess is that they’re using a combination of trying to figure out the need for JavaScript execution on a given page, coupled with trust/authority metrics to decide whether (and with what priority) to render a page with JS.

Run a test, get publicity

I have a hypothesis that I would love to see someone test: That it’s possible to get a page indexed and ranking for a nonsense word contained in the served HTML, but not initially ranking for a different nonsense word added via JavaScript; then, to see the JS get indexed some period of time later and rank for both nonsense words. If you want to run that test, let me know the results — I’d be happy to publicize them.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!