Archives by date

You are browsing the site archives by date.

Magento and Robots.txt

For all new Magento sites be sure to include a robots.txt.  There’s a major problem with robots spidering the search results.  It creates a huge server resource issue as well as an SEO issue.  Feel free to edit/comment on this as Magento evolves and things need to be added/removed.


## robots.txt for Magento Community and Enterprise
## http://turnkeye.com/blog/optimize-robots-txt-for-magento/

## GENERAL SETTINGS

## Enable robots.txt rules for all crawlers
User-agent: *

## Crawl-delay parameter: number of seconds to wait between successive requests to the same server.
## Set a custom crawl rate if you're experiencing traffic problems with your server.
# Crawl-delay: 30

## Magento sitemap: uncomment and replace the URL to your Magento sitemap file
# Sitemap: http://www.example.com/sitemap/sitemap.xml

## DEVELOPMENT RELATED SETTINGS

## Do not crawl development files and folders: CVS, svn directories and dump files
Disallow: /CVS
Disallow: /*.svn$
Disallow: /*.idea$
Disallow: /*.sql$
Disallow: /*.tgz$

## GENERAL MAGENTO SETTINGS

## Do not crawl Magento admin page
Disallow: /admin/

## Do not crawl common Magento technical folders
Disallow: /app/
Disallow: /downloader/
Disallow: /errors/
Disallow: /includes/
Disallow: /js/
Disallow: /lib/
Disallow: /media/css/
Disallow: /media/css_secure/
Disallow: /media/customer/
Disallow: /media/downloadable/
Disallow: /media/favicon/
Disallow: /media/import/
Disallow: /media/js/
Disallow: /media/sales/
Disallow: /media/tmp/
Disallow: /media/wysiwyg/
Disallow: /media/xmlconnect/
Disallow: /pkginfo/
Disallow: /shell/
Disallow: /skin/
Disallow: /var/

## Do not crawl common Magento files
Disallow: /api.php
Disallow: /cron.php
Disallow: /cron.sh
Disallow: /error_log
Disallow: /get.php
Disallow: /install.php
Disallow: /LICENSE.html
Disallow: /LICENSE.txt
Disallow: /LICENSE_AFL.txt
Disallow: /README.txt
Disallow: /RELEASE_NOTES.txt

## MAGENTO SEO IMPROVEMENTS

## Do not crawl sub category pages that are sorted or filtered.
Disallow: /*?dir*
Disallow: /*?dir=desc
Disallow: /*?dir=asc
Disallow: /*?limit=all
Disallow: /*?mode*

## Do not crawl 2-nd home page copy (example.com/index.php/). Uncomment it only if you activated Magento SEO URLs.
## Disallow: /index.php/

## Do not crawl links with session IDs
Disallow: /*?SID=

## Do not crawl checkout and user account pages
Disallow: /checkout/
Disallow: /onestepcheckout/
Disallow: /customer/
Disallow: /customer/account/
Disallow: /customer/account/login/

## Do not crawl seach pages and not-SEO optimized catalog links
Disallow: /catalogsearch/
Disallow: /catalog/product_compare/
Disallow: /catalog/category/view/
Disallow: /catalog/product/view/

## SERVER SETTINGS

## Do not crawl common server technical folders and files
Disallow: /cgi-bin/
Disallow: /cleanup.php
Disallow: /apc.php
Disallow: /memcache.php
Disallow: /phpinfo.php

## PROJECT SPECIFIC FILES AND FOLDERS

## Do not crawl these files and folders that are developer created
#Disallow: /staging-only/
#Disallow: /m13apc.php

## IMAGE CRAWLERS SETTINGS

## Extra: Uncomment if you do not wish Google and Bing to index your images
# User-agent: Googlebot-Image
# Disallow: /
# User-agent: msnbot-media
# Disallow: /

Oral and Maxillofacial Surgical Consultants

Oral and Maxillofacial Surgical Consultants

Site redesign for Oral and Maxillofacial Surgical Consultants.

http://www.omscmn.com/

  • WordPress, using Underscores Starter Theme
  • Various Plugins Used
    • Advanced Custom Fields
    • ACF Repeater
    • WordPress SEO by Yoast
    • Gravity Forms
    • The Events Calendar
    • Custom Menu Wizard
    • Responsive Lightbox
  • Responsive
  • HTML5, CSS3, Google Webfonts (the usual)
  • Launched 09/12/2013
Northland Capital

Northland Capital

A decent redesign here, got a little whacky with a lot of changes, but still pretty solid.

  • Responsive – Boilerplate-ish
  • Concrete5 – Composer, Global Blocks
  • Cycle2
  • Composer style News, Home Slider, Inventory
  • Inventory w/Designer Gallery + FancyBox2 galleries
  • Some custom forms (Online Payments, Payment Calculator)
  • Twitter feed with Twitter API and some js for display

www.northlandcapital.com

RGS Financial

A very simple redesign, by some firm in Texas. M13 chopped into c5. Nothing too special… Composer style testimonials for the right columns. A custom employment external form. I’m not even going to take a screenshot of it.