Search Engine Optimization (SEO) Crawl Terms


If you or an out-sourced SEO “agency” ever crawled your website for an SEO audit, the following terms will eventually come up. Get to know them and how they relate to your website. Its crucial as all of them affect SEO rankings.

Title Tag: Title tags—technically called title elements define the title of a document. Title tags are often used on search engine results pages (SERPs) to display preview snippets for a given page, and are important both for SEO and social sharing. Keep your titles under 55 characters, and you can expect at least 95% of your titles to display properly. Keep in mind that search engines may choose to display a different title than what you provide in your HTML. Titles in search results may be rewritten to match your brand, the user query, or other considerations.

Meta description: Meta descriptions are HTML attributes that provide concise explanations of the contents of web pages. Meta descriptions are commonly used on search engine result pages (SERPs) to display preview snippets for a given page. Optimal Length for Search Engines ~ Roughly 155 Characters. These short paragraphs is your opportunity to advertise content to searchers and to let them know exactly whether the given page contains the information they’re looking for. Tailor the Meta description to each webpage instead of a generic description.

HTTP Status Code: HyperText Transfer Protocol (or HTTP) response status codes are returned whenever search engines or website visitors make a request to a web server. These three-digit codes indicate the response and status of HTTP requests.

Referrer: The HTTP referrer is an HTTP header field that identifies the address of the webpage that linked to the resource being requested. By checking the referrer, the new webpage can see where the request originated.

Link Count: High link count is good. Make sure some of the links go outside your website to other resources.

Content-Type Header: text/html; charset=UTF-8 is good! Choose UTF-8 for all content and consider converting any content in legacy encodings to UTF-8. A Unicode-based encoding such as UTF-8 can support many languages and can accommodate pages and forms in any mixture of those languages, etc.

4XX (Client Error): If you get one of these errors, the description says that this is either bad or a broken link on your page. Find the URLs that cause the error.

5XX (Server Error): This group of HTTP status codes indicates that the server is aware that it is on error or is incapable of performing the request. The server response usually includes an explanation of the error situation and if it is a temporary or permanent condition.

Title Missing or Empty: This is a no-brainer. You need to have Titles for each webpage on the website.

URLs with Duplicate Page Content: Duplicate page content is bad. Never have duplicate page content or Google will penalize!

Duplicate Page Title: Same with Duplicate Page Titles. No duplicate page titles Google will penalize.

URLs with Duplicate Title Tags: No duplicate URLs or Title Tags. Google will penalize.

Long URL: Keep them short. Never long URLs.

Overly-Dynamic URL: A URL that results from the search of a database-driven Web site or the URL of a Web site that runs a script. In contrast to static URLs, in which the contents of the Web page do not change unless the changes are coded into the HTML, dynamic URLs are generated from specific queries to a site’s database.

301 (Permanent Redirect): Redirection is the process of forwarding one URL to a different URL.

302 (Temporary Redirect): Redirection is the process of forwarding one URL to a different URL.

Meta Refresh: Generally it’s not good for SEO, as in it doesn’t really pass link equity. So, if the crawl test says you don’t have any, then that’s a good thing.

Title Element Too Short: Don’t have them too short (just like the rapper!).

Title Element Too Long: Don’t have them too long <insert funny joke here>.

Too Many On-Page Links: Keep them low as to not look like spam.

Missing Meta Description Tag: Basic thing. Need them on every page. The description should optimally be between 150-160 characters.

Search Engine blocked by robots.txt: All pages are open (indexed) to Google, Yahoo, Bing, etc., to search your webpages. You MAY want certain webpages blocked due and not crawled by search engines, but that’s up to you and the content you choose to block.

Blocked by meta-robots: Some might blocked and some might not be blocked. Sometimes a meta robots block comes from your CMS or something in your header. Always check on the pages that show as blocked to see if there’s any noindex sort of code in those pages.

Meta Robots Tag: The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website. In most cases, meta robots with parameters “noindex, follow” should be employed as a way to to restrict crawling or indexation.

Blocking Google: If all are No then its good! You do not want to block Google as its good for content to be seen.

Blocking Yahoo: If all are No then its good! You do not want to block Yahoo as its good for content to be seen.

Blocking Bing: If all are No then its good! You do not want to block Bling as its good for content to be seen.

Does Yahoo Need a Smartphone or Tablet Now?

What about a Yahoo branded smartphone or Tablet? Jump on that train. Drive that Yahoo mobile strategy.

Google has one (Nexus 7 + Android), Bing has one (Windows Phone + Surface Tablet), Amazon has one (Tablet + rumor phone), Barnes and Noble (Tablet), Facebook (rumor to have phone in works).

Yahoo can drive their mobile ecosystem through a Tablet or Smartphone. Get an ODM (like FOXCONN) in Taiwan to design one, and get a mobile carrier to carry it. Your not going to out build a Nexus 7 or a Windows phone so make it run on the latest Andriod and play off their strengths. Yahoo could subsidize a smaller mobile carrier (T mobile) and make the killer feature a 5GB DATA PLAN for $20/MONTH. That’s the feature differentiator. A cheap data plan that is irresistible! But make that cheap data plan ONLY for the Yahoo smartphone or Tablet.

Increase Your Blog Traffic!

With Google + getting more relevant in Google’s search traffic ranking, its crucial that you share your posts via Google+ (in addition to Twitter and Facebook). Don’t miss this easy opportunity to leverage additional traffic or just increase brand awareness. LinkedIn also has this feature (status updates) that you can easily link your blog posts. These content sharing platforms also increase your chances of going viral as well as interacting with your own community. Don’t forget to exploit #hashtags in your status updates! Consistently using this strategy every time you post to your blog will increase your followers and drive traffic back to your site.

Stumbleupon: Dont Call It a Comeback!

Stumbleupon is still driving FIERCE traffic folks. In fact, StumbleUpon is now the biggest traffic driver among social media websites in the US, according to global web analytics service StatCounter. In fact, Its now a bigger traffic driver than Facebook. StumbleUpon is rolling out a new version on Sept. 1st. Watch for killer changes. Stumbleupon unseated Facebook at the top during June 2011, according to the latest StatCounter social media data. StatCounter tracks hits to over 3 million websites, and its social media data is gathered by analyzing every hit referred by a social media site.

Getting Links To Your Site

  • Reciprocal links. These links are obtained by contacting other website owners and trading links with them. Google and other search engines give very little value to these types of links anymore, particularly if your links are to and from an unrelated site. Weak spammy links from ad sites or content farms may decrease search engine value with the recent Google Panda update.
  • Directory submissions. There are thousands of niche and general interest directories on the internet that will allow you to submit a link to your website. Some are free and some are not. As with reciprocal links, the closer the directory is related to your site’s topic, the better.
  • Article submissions are another fairly simple way of obtaining non-reciprocal links to your site. Simply write (or hire someone to write) a good article and submit it for free to one of the top QUALITY article directories on the web.
  • Write valuable quality content that other site owners will want to link to on their own.
  • Competitive Intelligence – Analyze the back links of your competitors. Try to get links from the same sites.
  • Submit to Verticals/Topical Guides (local libraries, local chamber of commerce, etc.)
  • Use Blogs and RSS – Google loves blogs
  • Create Podcasts or just put up plain old Screenshots – Bloggers love posting them

Facebook’s No. 1 Ad Revenue


Social Network’s $2.19 Billion Puts It at Top of the Digital Heap, Leaping Google, Yahoo. Google will make $1.15 billion in display ads for 2011, down 10% from an earlier estimate, while Yahoo is expected to bring in $1.62 billion from banner ads, down less than 2% from an earlier analysis. Source: eMarketer research.

Search Engines Are Where Your Potential Customers Live

1/3 of US consumers spend three or more hours online every day. Source: The Media Audit, Oct. 2010

The average US Internet user views 2,750 web pages per month. Source: The Nielsen company, Jan. 2011

10.3 Billion Searches are conducted every month on Google. Source: Comscore, Aug. 2010

Search Engine

70% of the links search users click on are organic – not paid. Source: Marketing Sherpa, Feb. 2007

Search engine websites are the most visited websites with 77% of smartphone users citing this, followed by social networking, retail and video sharing websites. Source: Ipsos OTX, an independent market research firm, 2010

Nine out of ten smartphone searches results in an action (purchasing, visiting a business, etc.). Source: Ipsos OTX, an independent market research firm, 2010