Monthly Archives: February 2014

3 tips to find hacking on your site, and ways to prevent and fix it

Google shows this message in search results for sites that we believe may have been compromised.You might not think your site is a target for hackers, but it’s surprisingly common. Hackers target large numbers of sites all over the web in order to exploit the sites’ users or reputation.

One common way hackers take advantage of vulnerable sites is by adding spammy pages. These spammy pages are then used for various purposes, such as redirecting users to undesired or harmful destinations. For example, we’ve recently seen an increase in hacked sites redirecting users to fake online shopping sites.

Once you recognize that your website may have been hacked, it’s important to diagnose and fix the problem as soon as possible. We want webmasters to keep their sites secure in order to protect users from spammy or harmful content.

3 tips to help you find hacked content on your site

  1. Check your site for suspicious URLs or directories
    Keep an eye out for any suspicious activity on your site by performing a “site:” search of your site in Google, such as [site:example.com]. Are there any suspicious URLs or directories that you do not recognize?

    You can also set up a Google Alert for your site. For example, if you set a Google Alert for [site:example.com (viagra|cialis|casino|payday loans)], you’ll receive an email when these keywords are detected on your site.

  2. Look for unnatural queries on the Search Queries page in Webmaster Tools
    The Search Queries page shows Google Web Search queries that have returned URLs from your site. Look for unexpected queries as it can be an indication of hacked content on your site.

    Don’t be quick to dismiss queries in different languages. This may be the result of spammy pages in other languages placed on your website.

    Example of an English site hacked with Japanese content.
  3. Enable email forwarding in Webmaster Tools
    Google will send you a message if we detect that your site may be compromised. Messages appear in Webmaster Tools’ Message Center but it’s a best practice to also forward these messages to your email. Keep in mind that Google won’t be able to detect all kinds of hacked content, but we hope our notifications will help you catch things you may have missed.

Tips to fix and prevent hacking

  • Stay informed
    The Security Issues section in Webmaster Tools will show you hacked pages that we detected on your site. We also provide detailed information to help you fix your hacked site. Make sure to read through this documentation so you can quickly and effectively fix your site.
  • Protect your site from potential attacks
    It’s better to prevent sites from being hacked than to clean up hacked content. Hackers will often take advantage of security vulnerabilities on commonly used website management software. Here are some tips to keep your site safe from hackers:
    • Always keep the software that runs your website up-to-date.
    • If your website management software tools offer security announcements, sign up to get the latest updates.
    • If the software for your website is managed by your hosting provider, try to choose a provider that you can trust to maintain the security of your site.

We hope this post makes it easier for you to identify, fix, and prevent hacked spam on your site. If you have any questions, feel free to post in the comments, or drop by the Google Webmaster Help Forum.

If you find suspicious sites in Google search results, please report them using the Spam Report tool.

Posted by Megumi Hitomi, Japanese Search Quality Team

Continue reading

Posted in beginner, general tips, hacked sites, intermediate | Comments Off

8 Custom Reports from the Google Analytics Solutions Gallery

The following is a guest post from Rachelle Maisner, who recently transitioned from Sr Analyst to Account Manager at Digitaria, a Google Analytics Certified Partner.
New analysts have it easy these days. Back in my day, we have to walk uphill in the snow both ways to get decent enough web reporting. My first blush with web analytics came upon me when I was a marketing coordinator for an advertising agency several years ago. I got the hand-me-down grunt work of pulling stats for one of our client’s websites using server logs. Server logs, people. It was painfully slow, and gut-wrenchingly inefficient. So for the sake of my sanity, I started looking into better solutions, and I knew if it could help the client out with more meaningful reporting, that would make me look really good. When I found a solution I liked, I needed to pitch the client for their buy in. That conversation went something like… “I found this great tool, and it’s free- we can install it on your website and try out this fast new reporting. It’s called Google Analytics.”
Since then, there are now so many fantastic resources available to budding young analysts. From the Analysis Exchange to Avinash’s own Market Motive courses, not to mention GA’s recently revamped Analytics Academy, there’s a wealth of quality education and training just a click away to anyone who’s willing to learn. 
I’m blogging to tell you all about one of my absolute favorite new resources– a tremendous goldmine of knowledge sharing unlike anything else this industry has ever seen– Google Analytics’ very own Solutions Gallery.
The Solutions Gallery is a free and public platform that allows users to share custom reports, segments and dashboards. It’s invaluable resource not only for those that are new to digital analytics, but also for analytics veterans looking for fresh ideas and new approaches. I mean, wow, you can download reports and dashboards from experts all over the globe and instantly apply them to your own Google Analytics account. 

I was so excited about the Solutions Gallery that I uploaded 8 custom reports of my own to share with the community, and in about a month I had over 1,600+ imports. 
I have received awesome feedback and gratitude for the custom reports I created, so I am absolutely thrilled to be able to share them here on the Google Analytics blog and showcase them to a wider audience. I hope you find these helpful and I hope they encourage you to not only get more from your data, but to upload some of your own solutions to the Gallery.
All my custom reports are organized into four categories. These categories are based on the ABC’s of analytics, plus D for bonus points: Acquisition, Behavior, Conversion, and Diagnostics.
A is for Acquisition
Visits and Goal Conversion by Traffic Source: Take your traffic source reports one step further by understanding volume & conversion by each channel. This is one way to see how your best visitors are getting to your site. I recommend setting up a goal for “engaged visits”, for this custom report and some of the following reports. When you import this custom report, change Goal One to your engaged visits goal, or another significant KPI configured as a goal.
B is for Behavior
Page Effectiveness: Ever ponder the question, “How is my content doing?” This custom report provides page-level performance, allowing you to discover your top and bottom performing pages using various traffic and engagement metrics.
Social Sharing: A four-tab custom report chock full of site-to-social reporting. Tab 1 is the Shared Content Trend, showing how top pages are shared to social networks over time. Tab 2 is Top Shared Content by Network, a first step to discovering what content works for specific channels. Tab 3 is a report on Socially Engaged Visitors, providing a quick profile of visitors that engage in social sharing links. And finally, Tab 4 is Social Outcomes and Conversions, tying social engagement to site goals.
C is for Conversion
Simple E-Commerce Report: A starting point for trending revenue or purchases against visits, with a traffic sources breakdown.
PPC Campaign Goal Performance: Analyze paid search performance against goal conversion by search engine. Change goal one completions to your engaged visits goal. This report filters for Google campaigns. To filter for Bing change the source filter for “Bing” or delete the filter to include all search engines.
PPC Keywords: Get a paid keyword report with traffic volume, CPC, goal conversions, and cost per conversion.
D is for Diagnostics
Page Timing: Use this custom report to QA page load timing and reveal problem pages. Switch from the “data” table view to the “comparison” table view, and compare load time to bounce rate, allowing you to view the bounce rate for each page against the site average.
Internal and External 404 Report: A custom report to help resolve 404 errors. Includes two report tabs. Tab 1: bad inbound links, and Tab 2: bad internal links. Be sure to change the filter for “page title” to the page title used on your site’s 404 page.
Posted by Rachelle Maisner, Account Manager at Digitaria, a Google Analytics Certified Partner

Continue reading

Posted in Features, Partners | Comments Off

New: AdWords Reports for App download campaigns

Today we’re happy to announce a deeper integration between AdWords and Google Analytics for Mobile Apps that will help advertisers make faster and better decisions about marketing their apps.
To put it simply: link your AdWords and Google Analytics accounts and enable auto-tagging, and you’ll start receiving a new set of detailed reports on things like day parts, destination URLs and keyword positions. These automatic reports show exactly how your search and display campaigns are performing and offer rich insights into the kind of users they’re driving towards to Google Play. 
Any user of both AdWords and Google Analytics can have this set of reports by just enabling auto-tagging. We handle the rest, so you can focus on optimizing rather than manually tagging AdWords campaigns.

These new reports can be found under the Acquisition menu for Google Analytics App Views. They’ll become visible to everyone over the next few days.
Insights for display & search campaigns
These new reports cover both display and search campaigns. You can: 
  • Check the Campaigns report to better understand users being driven into your app, and see how they use your app. 
  • Find out from the Day Parts report when users are interacting with your campaigns.
  • Use Search reports to find out which keywords and search queries are acquiring the most new users.
Click image for full-sized version
A step towards measuring lifetime value of your customers
Ad campaigns should help you find the best customers. These new reports go a long way towards identifying them. Whether you track in-app revenue or specific goal conversions, you’ll be able to tie user quality to the campaign that brought them to your app.
One of our early Beta testers was Nubee, a Singapore-based game development studio. They shared their experience with us:
We were satisfied that we could track which keywords attributed to sales. Using this data, we were also able to modify the download page.” – Shizuka Watanbe, Head of PR, Nubee
We hope you’ll find these new reports useful. You can get them running by linking accounts and enabling auto-tagging today.

Posted by Rahul Oak, Product Manager, Google Analytics for Apps

Continue reading

Posted in AdWords, Announcements, mobile | Comments Off

Bloggerwan Hadirkan livestreaming LIVECHATkustik NIDJI dan FRIDAY

Bloggerwan mempersembahkan #livestreaming #LIVECHATkustik NIDJI Feat. FRIDAY Sambil melihat penampilan NIDJI dan juga FRIDAY, kamu juga bisa mengajukan pertanyaan kepada mereka lewat official thread #LIVECHATkustik bareng NIDJI. Kamu belum mengenal NID… Continue reading

Comments Off

Efek Snowden Terhadap DuckDuckGo

Lintas Posting dari Bloggerwan. Selain dapat dimanfaatkan untuk mencari informasi secara online, mesin telusur seperti Google dan Bing ternyata menyimpan data para penggunanya. Bagi yang ingin menghindari hal tersebut, mesin telusur DuckDuckGo bi… Continue reading

Comments Off

Ensuring Data Accuracy with a Tag Management Policy

The following is a guest post from GACP Michael Loban, CMO at InfoTrust.
The quality of the website analytics data we have is directly related to the tag management processes adopted by an organization. Most likely, you can remember days when the following incidents may have occurred:
  1. You find that one (or several) of the pages on your site is missing Google Analytics, or some pages had Google Analytics deployed twice causing duplicate pageviews and inflating traffic.
  2. Google Analytics custom variables were inconsistent or missing on some portions of the site, leading to data quality issues.
  3. An unauthorized marketing tag was piggybacking off of another tag.
  4. One of the tags on an international site you managed did not follow the new EU Cookie Laws related to privacy.
Adopting a Tag Management System like Google Tag Manager is a great way to go, but having a great tool to organize and deploy your tags is often not enough. You still need a system, a process, and ongoing review. Here are the steps for creating a tag management policy for your company:
1. Know where you are – what tags are currently firing, where and how? Whether you have a small site with a few hundred pages or an international publication with thousands of pages, it is important to assess your current tag deployment. 
Can you say, with 100% confidence, that your analytics tag are located on every page?  Are you sure the cookies set by your analytics tag/tool are accurate and not over-writing each other?
Regardless of whether you are confident or not, I suggest using a tool like TagInspector.com (Tag Inspector is an InfoTrust product). It will help you locate:
  1. All the tags on your site, split up by specific pages’ tags, and even pages they are missing from.
  2. Cookies set by various tags and what pages they are set on.
  3. How the tag is deployed – through a tag management system or directly from a page source.
  4. Instances of tag piggybacking – one tag being loaded by another tag.
Here is a screenshot from an example scan. It shows how tags load (commonly referred to as tag hierarchy). We have removed the website URL, but as you can see there are instances when Google Analytics is being loaded by the TMS, and instances where Google Analytics is being loaded directly from the source of the page. 

2. Document all approved tags. The average enterprise website might have 25-50 marketing tags. Not all of them have to be present across all pages. However, even if you are considering moving to a Tag Management System, or already are using one, it is not a bad idea to have the following documented and categorized:
  1. Tag name and functionality
  2. Pages or the category pages the tag needs to be on
  3. Information collected through the tag about visitors (cookies set)
  4. Firing rules
Check out Tagopedia – a wiki of tags to learn more about the many different types of tags.
3. Consider the implementation of a Tag Management System. There is a reason this is step three, and not step one or two. A lot of companies jump to this step first, thinking that a new technology will miraculously make all tagging issues disappear. The first step in moving to a TMS is knowing what tags you need to keep, and where they are or how they are loaded on your site so you can remove them from the source after the tag management system is deployed.

When considering the implementation of a tag management system, think about your team. Every website of a TMS vendor says you will no longer need your IT team to make changes to the tags thus simplifying and expediting the process. I have met plenty of marketers who do not want anything to do with a TMS. Even though you will free up your IT resources, you will still need a person or team with the technical training to manage your tags. 

Naturally, your first step in evaluating Tag Management vendors should be outlining what features you really need. Google Tag Manager is free, and is one of the few TMS systems that works for both mobile websites and native mobile applications. 
NOTE:  If you do decide to migrate to a TMS or if you have already done so, you still should scan all the pages across your site to ensure that your tags fire correctly, such as, once per page for analytics tags – and only from your TMS. You certainly want to avoid having a tag in the source of your page and inside a TMS – this will inflate your data and cause data quality issues.
4. Run ongoing site audits to ensure correct tags are deployed across correct pages. Ideally, this will only serve as the insurance. However, ongoing site scans or audits can help you avoid the moments when you realize you did not capture AdWords conversions because your GA or AdWords conversion tag was removed from the conversion page. Keep in mind certain tags might only fire when a user looks at your website on a mobile device, and your scan might need to simulate different user agents.  Doing this manually for all the sites you manage, or across one very large site, can be quite challenging. Again, TagInspector.com can help speed up this process and dramatically reduce the effort required. Here is an example screenshot of the scanning options:

5. Think ahead – will you be able to innovate? Complete lock down is in nobody’s best interests. What happens if there is a new platform for A/B testing that you would like to try? How long will it take you to get the tag approved, implemented on your site, verify its performance, and launch a campaign? Keep innovation in mind and make it relatively easy for marketers in your company to adopt new technologies.

One way to go about this is having an application that needs to be completed and approved prior to implementing a new tag. This will help you ensure only tags that meet company standards are implemented on your site. 
At the end of the day, tag deployment and data collection will only get more complex. If you do not have any process for managing your tags, it is time to start. If you have some kind of process, perhaps it is time for optimization. Get all the stakeholders in the room, and decide who will be your tag management team, and what the first step will be to ensure tag accuracy. You can’t do analysis if the data isn’t accurate. And your data won’t be accurate if your marketing tags aren’t implemented correctly. 
If you would like to learn more about implementing a tag management policy, we would like to invite you to attend a free webinar on March 26th at 1:00PM EST where we will discus items outlined in this post, and a lot more. 
Posted by Michael Loban, CMO at Google Analytics Certified Partner InfoTrust

Continue reading

Posted in Advanced Topics, Partners, Upcoming Events | Comments Off

Watchfinder clocks 1300% ROI using precision Remarketing with Google Analytics

Watchfinder is a leading UK retailer of premium, pre-owned watches. The company was founded in 2002 as an online-only store selling watches from more than 80 premier manufacturers. Today, it has an annual turnover of £25 million and has recently opened a flagship boutique in the London Royal Exchange.

Counting the hours 
Considering the average order value on Watchfinder’s site is over £3,500, the company found buying decisions tended to take time, often spanning weeks or months. In fact, less than 1% of visitors were completing purchases on their first site visit. Watchfinder’s challenge was to re-engage and also maintain a conversation with these visitors, encouraging them to return and make an order. In addition to driving customers back to its site, Watchfinder also wanted to encourage customers to visit its new physical boutique in the London Royal Exchange.
A moment to reconnect
Watchfinder’s agency Periscopix – a Google Analytics Certified Partner – suggested Remarketing with Google Analytics as a great way to reconnect with users. Remarketing with Google Analytics allows advertisers to tap into valuable insights about website visitors who show an interest in products, identify the most relevant audiences, and run ads across the Google Display Network that are tailored to that audience using the industry’s most powerful segmentation capabilities.
Periscopix created 20 highly focused lists of visitors who demonstrated intent but did not purchase. Specifically, lists were based on various aspects of user context such as location, language, and what stage of the purchase funnel they were in. On-site behavior helped establish groups that had spent a certain amount of time on the site or had viewed a certain number of pages. Other lists were created around users who had viewed a specific watch brand on the site.
Additionally, traffic performance analysis across a variety of GA dimensions revealed that certain ISP’s in the London financial district yielded traffic with much higher engagement and above average conversion rates. As a result, Periscopix designed segments around investment banks like JPMorgan and Goldman Sachs to engage with employees at these companies.

Google Analytics’ functionality enabled Periscopix to convey tailored messages to these key groups of interested consumers. For example, London-based users were retargeted with ads encouraging visits to the new London store, while visitors to the .co.uk site from France were retargeted with ads promoting the French site. 
Time well spent
Thanks to clear reporting in Google Analytics, it’s been easy to see the impressive results from Watchfinders’ remarketing campaign. Six months in, Periscopix reveals the return on investment is 1300%. Average order value on the site has also increased by 13%, resulting in 34% lower CPAs than Watchfinder’s non-brand search campaigns. 
Across all tactics used, the remarketing list that produced the highest conversion rates, both in terms of goals and transactions, was made up of visitors who browsed for 10 minutes or more on their initial site visit without purchasing.
Watchfinder has also seen early success with Remarketing with Google Analytics across the Google Display Network. As a result, they are excited to increase investment in this area going forward.
Be sure to check out the whole case study here.

Posted by the Google Analytics Team

Continue reading

Posted in Case Study | Comments Off

Infinite scroll search-friendly recommendations

Webmaster Level: Advanced

Your site’s news feed or pinboard might use infinite scroll—much to your users’ delight! When it comes to delighting Googlebot, however, that can be another story. With infinite scroll, crawlers cannot always emulate manual user behavior–like scrolling or clicking a button to load more items–so they don’t always access all individual items in the feed or gallery. If crawlers can’t access your content, it’s unlikely to surface in search results.

To make sure that search engines can crawl individual items linked from an infinite scroll page, make sure that you or your content management system produces a paginated series (component pages) to go along with your infinite scroll.


Infinite scroll page is made “search-friendly” when converted to a paginated series — each component page has a similar <title> with rel=next/prev values declared in the <head>.

You can see this type of behavior in action in the infinite scroll with pagination demo created by Webmaster Trends Analyst, John Mueller. The demo illustrates some key search-engine friendly points:

  • Coverage: All individual items are accessible. With traditional infinite scroll, individual items displayed after the initial page load aren’t discoverable to crawlers.
  • No overlap: Each item is listed only once in the paginated series (i.e., no duplication of items).

Search-friendly recommendations for infinite scroll

  1. Before you start:
    • Chunk your infinite-scroll page content into component pages that can be accessed when JavaScript is disabled.
    • Determine how much content to include on each page.
      • Be sure that if a searcher came directly to this page, they could easily find the exact item they wanted (e.g., without lots of scrolling before locating the desired content).
      • Maintain reasonable page load time.
    • Divide content so that there’s no overlap between component pages in the series (with the exception of buffering).

  2. The example on the left is search-friendly, the right example isn’t — the right example would cause crawling and indexing of duplicative content.

  3. Structure URLs for infinite scroll search engine processing.
    • Each component page contains a full URL. We recommend full URLs in this situation to minimize potential for configuration error.
      • Good: example.com/category?name=fun-items&page=1
      • Good: example.com/fun-items?lastid=567
      • Less optimal: example.com/fun-items#1
      • Test that each component page (the URL) works to take anyone directly to the content and is accessible/referenceable in a browser without the same cookie or user history.
    • Any key/value URL parameters should follow these recommendations:
      • Be sure the URL shows conceptually the same content two weeks from now.
        • Avoid relative-time based URL parameters:
          example.com/category/page.php?name=fun-items&days-ago=3
      • Create parameters that can surface valuable content to searchers.
        • Avoid non-searcher valuable parameters as the primary method to access content:
          example.com/fun-places?radius=5&lat=40.71&long=-73.40
  4. Configure pagination with each component page containing rel=next and rel=prev values in the <head>. Pagination values in the <body> will be ignored for Google indexing purposes because they could be created with user-generated content (not intended by the webmaster).
  5. Implement replaceState/pushState on the infinite scroll page. (The decision to use one or both is up to you and your site’s user behavior). That said, we recommend including pushState (by itself, or in conjunction with replaceState) for the following:
    • Any user action that resembles a click or actively turning a page.
    • To provide users with the ability to serially backup through the most recently paginated content.
  6. Test!

Written, reviewed, or coded by John Mueller, Maile Ohye, and Joachim Kupke

Continue reading

Posted in advanced, crawling and indexing | Comments Off

Faceted navigation best (and 5 of the worst) practices

Webmaster Level: Advanced

Faceted navigation, such as filtering by color or price range, can be helpful for your visitors, but it’s often not search-friendly since it creates many combinations of URLs with duplicative content. With duplicative URLs, search engines may not crawl new or updated unique content as quickly, and/or they may not index a page accurately because indexing signals are diluted between the duplicate versions. To reduce these issues and help faceted navigation sites become as search-friendly as possible, we’d like to:


Selecting filters with faceted navigation can cause many URL combinations, such as http://www.example.com/category.php?category=gummy-candies&price=5-10&price=over-10

Background

In an ideal state, unique content — whether an individual product/article or a category of products/articles —  would have only one accessible URL. This URL would have a clear click path, or route to the content from within the site, accessible by clicking from the homepage or a category page.

Ideal for searchers and Google Search

  • Clear path that reaches all individual product/article pages


    On the left is potential user navigation on the site (i.e., the click path), on the right are the pages accessed.

  • One representative URL for category page
    http://www.example.com/category.php?category=gummy-candies

    Category page for gummy candies

  • One representative URL for individual product page
    http://www.example.com/product.php?item=swedish-fish

    Product page for swedish fish

Undesirable duplication caused with faceted navigation

  • Numerous URLs for the same article/product

    Canonical Duplicate
    example.com/product.php? item=swedish-fish example.com/product.php? item=swedish-fish&category=gummy-candies&price=5-10

    Same product page for swedish fish can be available on multiple URLs.

  • Numerous category pages that provide little or no value to searchers and search engines)
    URL example.com/category.php? category=gummy-candies&taste=sour&price=5-10 example.com/category.php? category=gummy-candies&taste=sour&price=over-10
    Issues
    • No added value to Google searchers given users rarely search for [sour gummy candy price five to ten dollars].
    • No added value for search engine crawlers that discover same item (“fruit salad”) from parent category pages (either “gummy candies” or “sour gummy candies”).
    • Negative value to site owner who may have indexing signals diluted between numerous versions of the same category.
    • Negative value to site owner with respect to serving bandwidth and losing crawler capacity to duplicative content rather than new or updated pages.
    • No value for search engines (should have 404 response code).
    • Negative value to searchers.

Worst (search un-friendly) practices for faceted navigation

Worst practice #1: Non-standard URL encoding for parameters, like commas or brackets, instead of “key=value&” pairs.

Worst practices:
  • example.com/category?[category:gummy-candy][sort:price-low-to-high][sid:789]
    • key=value pairs marked with : rather than =
    • multiple parameters appended with [ ] rather than &
  • example.com/category?category,gummy-candy,,sort,lowtohigh,,sid,789
    • key=value pairs marked with a , rather than =
    • multiple parameters appended with ,, rather than &

Best practice:

example.com/category?category=gummy-candy&sort=low-to-high&sid=789

While humans may be able to decode odd URL parameters, such as “,,”, crawlers have difficulty interpreting URL parameters when they’re implemented in a non-standard fashion. Software engineer on Google’s Crawling Team, Mehmet Aktuna, says “Using non-standard encoding is just asking for trouble.” Instead, connect key=value pairs with an equal sign (=) and append multiple parameters with an ampersand (&).

Worst practice #2: Using directories or file paths rather than parameters to list values that don’t change page content.

Worst practice:
example.com/c123/s789/product?swedish-fish
(where /c123/ is a category, /s789/ is a sessionID that doesn’t change page content)

Good practice:

example.com/gummy-candy/product?item=swedish-fish&sid=789 (the directory, /gummy-candy/,changes the page content in a meaningful way)

Best practice:

example.com/product?item=swedish-fish&category=gummy-candy&sid=789 (URL parameters allow more flexibility for search engines to determine how to crawl efficiently)

It’s difficult for automated programs, like search engine crawlers, to differentiate useful values (e.g., “gummy-candy”) from the useless ones (e.g., “sessionID”) when values are placed directly in the path. On the other hand, URL parameters provide flexibility for search engines to quickly test and determine when a given value doesn’t require the crawler access all variations.

Common values that don’t change page content and should be listed as URL parameters include:

  • Session IDs
  • Tracking IDs
  • Referrer IDs
  • Timestamp

Worst practice #3: Converting user-generated values into (possibly infinite) URL parameters that are crawlable and indexable, but not useful in search results.

Worst practices (e.g., user-generated values like longitude/latitude or “days ago” as crawlable and indexable URLs):

  • example.com/find-a-doctor?radius=15&latitude=40.7565068&longitude=-73.9668408

  • example.com/article?category=health&days-ago=7

Best practices:

  • example.com/find-a-doctor?city=san-francisco&neighborhood=soma

  • example.com/articles?category=health&date=january-10-2014

Rather than allow user-generated values to create crawlable URLs  – which leads to infinite possibilities with very little value to searchers — perhaps publish category pages for the most popular values, then include additional information so the page provides more value than an ordinary search results page. Alternatively, consider placing user-generated values in a separate directory and then robots.txt disallow crawling of that directory.

  • example.com/filtering/find-a-doctor?radius=15&latitude=40.7565068&longitude=-73.9668408
  • example.com/filtering/articles?category=health&days-ago=7

with robots.txt:

User-agent: *
Disallow: /filtering/

Worst practice #4: Appending URL parameters without logic.

Worst practices:

  • example.com/gummy-candy/lollipops/gummy-candy/gummy-candy/product?swedish-fish
  • example.com/product?cat=gummy-candy&cat=lollipops&cat=gummy-candy&cat=gummy-candy&item=swedish-fish

Better practice:

example.com/gummy-candy/product?item=swedish-fish

Best practice:

example.com/product?item=swedish-fish&category=gummy-candy

Extraneous URL parameters only increase duplication, causing less efficient crawling and indexing. Therefore, consider stripping unnecessary URL parameters and performing your site’s “internal housekeeping”  before generating the URL. If many parameters are required for the user session, perhaps hide the information in a cookie rather than continually append values like cat=gummy-candy&cat=lollipops&cat=gummy-candy&

Worst practice #5: Offering further refinement (filtering) when there are zero results.

Worst practice:

Allowing users to select filters when zero items exist for the refinement.


Refinement to a page with zero results (e.g., price=over-10) is allowed even though it frustrates users and causes unnecessary issues for search engines.

Best practice

Only create links/URLs when it’s a valid user-selection (items exist). With zero items, grey out filtering options. To further improve usability, consider adding item counts next to each filter.


Refinement to a page with zero results (e.g., price=over-10) isn’t allowed, preventing users from making an unnecessary click and search engine crawlers from accessing a non-useful page.

Prevent useless URLs and minimize the crawl space by only creating URLs when products exist. This helps users to stay engaged on your site (fewer clicks on the back button when no products exist), and helps minimize potential URLs known to crawlers. Furthermore, if a page isn’t just temporarily out-of-stock, but is unlikely to ever contain useful content, consider returning a 404 status code. With the 404 response, you can include a helpful message to users with more navigation options or a search box to find related products.

Best practices for new faceted navigation implementations or redesigns

New sites that are considering implementing faceted navigation have several options to optimize the “crawl space” (the totality of URLs on your site known to Googlebot) for unique content pages, reduce crawling of duplicative pages, and consolidate indexing signals.

  • Determine which URL parameters are required for search engines to crawl every individual content page (i.e., determine what parameters are required to create at least one click-path to each item). Required parameters may include item-id, category-id, page, etc.
  • Determine which parameters would be valuable to searchers and their queries, and which would likely only cause duplication with unnecessary crawling or indexing. In the candy store example, I may find the URL parameter “taste” to be valuable to searchers for queries like [sour gummy candies] which could show the result example.com/category.php?category=gummy-candies&taste=sour. However, I may consider the parameter “price” to only cause duplication, such as category=gummy-candies&taste=sour&price=over-10. Other common examples:
    • Valuable parameters to searchers: item-id, category-id, name, brand
    • Unnecessary parameters: session-id, price-range
  • Consider implementing one of several configuration options for URLs that contain unnecessary parameters. Just make sure that the unnecessary URL parameters are never required in a crawler or user’s click path to reach each individual product!
    • Option 1: rel=”nofollow” internal links
      Make all unnecessary URLs links rel=“nofollow.” This option minimizes the crawler’s discovery of unnecessary URLs and therefore reduces the potentially explosive crawl space (URLs known to the crawler) that can occur with faceted navigation. rel=”nofollow” doesn’t prevent the unnecessary URLs from being crawled (only a robots.txt disallow prevents crawling). By allowing them to be crawled, however, you can consolidate indexing signals from the unnecessary URLs with a searcher-valuable URL by adding rel=”canonical” from the unnecessary URL to a superset URL (e.g. example.com/category.php?category=gummy-candies&taste=sour&price=5-10 can specify a rel=”canonical” to the superset sour gummy candies view-all page at example.com/category.php?category=gummy-candies&taste=sour&page=all).

    • Option 2: Robots.txt disallow
      For URLs with unnecessary parameters, include a /filtering/directory that will be robots.txt disallow’d. This lets all search engines freely crawl good content, but will prevent crawling of the unwanted URLs. For instance, if my valuable parameters were item, category, and taste, and my unnecessary parameters were session-id and price. I may have the URL:
      example.com/category.php?category=gummy-candies
      which could link to another URL valuable parameter such as taste:
      example.com/category.php?category=gummy-candies&taste=sour.
      but for the unnecessary parameters, such as price, the URL includes a predefined directory, /filtering/:
      example.com/filtering/category.php?category=gummy-candies&price=5-10
      which is then robots.txt disallowed
      User-agent: *
      Disallow: /filtering/

    • Option 3: Separate hosts
      If you’re not using a CDN (sites using CDNs don’t have this flexibility easily available in Webmaster Tools), consider placing any URLs with unnecessary parameters on a separate host — for example, creating main host www.example.com and secondary host, www2.example.com. On the secondary host (www2), set the Crawl rate in Webmaster Tools to “low” while keeping the main host’s crawl rate as high as possible. This would allow for more full crawling of the main host URLs and reduces Googlebot’s focus on your unnecessary URLs.
      • Be sure there remains at least one click path to all items on the main host.
      • If you’d like to consolidate indexing signals, consider adding rel=”canonical” from the secondary host to a superset URL on the main host (e.g. www2.example.com/category.php?category=gummy-candies&taste=sour&price=5-10 may specify a rel=”canonical” to the superset “sour gummy candies” view-all page, www.example.com/category.php?category=gummy-candies&taste=sour&page=all).
  • Prevent clickable links when no products exist for the category/filter.
  • Add logic to the display of URL parameters.
    • Remove unnecessary parameters rather than continuously append values.
      • Avoid
        example.com/product?cat=gummy-candy&cat=lollipops&cat=gummy-candy&item=swedish-fish)
    • Help the searcher experience by keeping a consistent parameter order based on searcher-valuable parameters listed first (as the URL may be visible in search results) and searcher-irrelevant parameters last (e.g., session ID).
      • Avoid
        example.com/category.php?session-id=123&tracking-id=456&category=gummy-candies&taste=sour
  • Improve indexing of individual content pages with rel=”canonical” to the preferred version of a page. rel=”canonical” can be used across hostnames or domains.
  • Improve indexing of paginated content (such as page=1 and page=2 of the category “gummy candies”) by either:
    • Adding rel=”canonical” from individual component pages in the series to the category’s “view-all” page (e.g. page=1, page=2, and page=3 of “gummy candies” with rel=”canonical” to category=gummy-candies&page=all) while making sure that it’s still a good searcher experience (e.g., the page loads quickly).
    • Using pagination markup with rel=”next” and rel=”prev” to consolidate indexing properties, such as links, from the component pages/URLs to the series as a whole.
  • Be sure that if using JavaScript to dynamically sort/filter/hide content without updating the URL, there still exists URLs on your site that searchers would find valuable, such as main category and product pages that can be crawled and indexed. For instance, avoid using only the homepage (i.e., one URL) for your entire site with JavaScript to dynamically change content with user navigation —  this would unfortunately provide searchers with only one URL to reach all of your content. Also, check that performance isn’t negatively affected with dynamic filtering, as this could undermine the user experience.
  • Include only canonical URLs in Sitemaps.

Best practices for existing sites with faceted navigation

First, know that the best practices listed above (e.g., rel=”nofollow” for unnecessary URLs) still apply if/when you’re able to implement a larger redesign. Otherwise, with existing faceted navigation, it’s likely that a large crawl space was already discovered by search engines. Therefore, focus on reducing further growth of unnecessary pages crawled by Googlebot and consolidating indexing signals.

  • Use parameters (when possible) with standard encoding and key=value pairs.
  • Verify that values that don’t change page content, such as session IDs, are implemented as standard key=value pairs, not directories
  • Prevent clickable anchors when products exist for the category/filter (i.e., don’t allow clicks or URLs to be created when no items exist for the filter)
  • Add logic to the display of URL parameters
    • Remove unnecessary parameters rather than continuously append values (e.g., avoid example.com/product?cat=gummy-candy&cat=lollipops&cat=gummy-candy&item=swedish-fish)
  • Help the searcher experience by keeping a consistent parameter order based on searcher-valuable parameters listed first (as the URL may be visible in search results) and searcher-irrelevant parameters last (e.g., avoid example.com/category?session-id=123&tracking-id=456&category=gummy-candies&taste=sour& in favor of example.com/category.php?category=gummy-candies&taste=sour&session-id=123&tracking-id=456)
  • Configure Webmaster Tools URL Parameters if you have strong understanding of the URL parameter behavior on your site (make sure that there is still a clear click path to each individual item/article). For instance, with URL Parameters in Webmaster Tools, you can list the parameter name, the parameters effect on the page content, and how you’d like Googlebot to crawl URLs containing the parameter.

    URL Parameters in Webmaster Tools allows the site owner to provide information about the site’s parameters and recommendations for Googlebot’s behavior.
  • Be sure that if using JavaScript to dynamically sort/filter/hide content without updating the URL, there still exists URLs on your site that searchers would find valuable, such as main category and product pages that can be crawled and indexed. For instance, avoid using only the homepage (i.e., one URL) for your entire site with JavaScript to dynamically change content with user navigation —  this would unfortunately provide searchers with only one URL to reach all of your content. Also, check that performance isn’t negatively affected with dynamic filtering, as this could undermine the user experience.
  • Improve indexing of individual content pages with rel=”canonical” to the preferred version of a page. rel=”canonical” can be used across hostnames or domains.
  • Improve indexing of paginated content (such as page=1 and page=2 of the category “gummy candies”) by either:
    • Adding rel=”canonical” from individual component pages in the series to the category’s “view-all” page (e.g. page=1, page=2, and page=3 of “gummy candies” with rel=”canonical” to category=gummy-candies&page=all) while making sure that it’s still a good searcher experience (e.g., the page loads quickly).
    • Using pagination markup with rel=”next” and rel=”prev” to consolidate indexing properties, such as links, from the component pages/URLs to the series as a whole.
  • Include only canonical URLs in Sitemaps.

Remember that commonly, the simpler you can keep it, the better. Questions? Please ask in our Webmaster discussion forum.

Written by Maile Ohye, Developer Programs Tech Lead, and Mehmet Aktuna, Crawl Team

Continue reading

Posted in advanced, crawling and indexing | Comments Off

Zillow uses Google Analytics Premium to make data-driven decisions

Google Analytics Premium lets Zillow grow and scale their company.
A host of functions at Zillow use Google Analytics every day… Marketing, business intelligence, design, engineering and usability are using it to drive product decisions, user experience decisions, and business decisions.”
- Jeremy Wacksman, VP Marketing at Zillow
Zillow is the home and real estate marketplace that helps people share vital information about home values, rentals, mortgages and a lot more. Zillow was founded in 2005 and now has over 110 million U.S. homes in its living database. (That name? A combination of “zillions of data points” and the pillows where happy homeowners rest their heads.)
Recently we sat down with Jeremy Wacksman, Zillow’s VP of Marketing, to learn how they’ve been using Google Analytics Premium to help them grow at such an amazing pace. Here’s what he told us:

This comment stands out: “As an Internet company that has reinvented itself as a mobile-first business, analytics across devices is a big challenge for us.”  
A lot of companies are reinventing themselves for mobile today, and we’ve been working hard to make sure Google Analytics Premium can help them measure all those new cross-device journeys. The goal, as always, is to help businesses gather meaningful data, easily discover insights that they can act upon to improve results and boost the bottom line.
Learn more about Google Analytics Premium here.

Posted by Adam Singer, Google Analytics Advocate

Continue reading

Posted in Case Study, Google Analytics Premium | Comments Off