Tips and Tricks for Sitecore optimization

Publishing 

• How does Sitecore publish of millions of items? 

The simple answer is: nothing special is done. The rationale is in the answer to this question:
“Why would you need to publish 1 million items all the time?” The first time you publish from
authoring to delivery and if you were to use the Full Publish option, then the answer is: “Yes,
this would take some time.” But after that, you only need to run publish added content
incrementally.

• If I publish a single item in a bucket, but its bucket folders are not published, what happens? 

Sitecore has added an extra pipeline step to the publishing process to detect if an item
requires its bucket folders to be published and will add them to the publishing queue as well
automatically. This also applies to items that are in workflow. There is no need to add in an
extra workflow action for this.

• What publishing should I be doing for Item Buckets? 

The answer to this question does not change because of Item Buckets.

Standard Web.Config Tweaks 

• You should periodically tweak the cache depending on how many items are in the content 

tree and how many similar searches have been processed.

Setup Tweaks 

• When you import a lot of content programmatically, you must truncate the PublishingQueue, History, and Event Queue tables in the Master and Web databases and rebuild the indexes on the database tables. If you don’t do this, the PublishingQueue, History, and EventQueue tables will get very large, slow down processing, and your Sitecore installation may not start. After clearing the tables, you must rebuild the index and run a smart publish instead of an incremental publish. 

Environment Tweaks

• If possible,e, disable the inbuilt Windows Search Index as well as any other indexer that is running on the computer that runs the index or on the web server itself. This index uses essential Disk I/O resources that Lucene.net needs. 
• Don’t run processes on the index to create a backup. The index should not be part of regular backup procedures. Chances are that the backup will be outdated if it is ever needed, so it is a waste of resources. 
• It is very important that you set up a SQL Maintenance plan that rebuilds your indexes. When you create a lot of content, index fragmentation will increase, especially with the bulk importation of content. The hotspots will be the Items, Versioned, Unversioned, Shared, Blobs, and Links tables. To be on the safe side, you should set rebuilds for every table. If you don’t do this, performance of the CMS will degrade. 

Importing Data Tweaks 

• When you import a lot of content into Sitecore, it is best to use the BulkUpdateContext class. When you have imported the content, rebuild the Lucene index. 
• If you import a lot of content, do it in batches of, for example, one thousand, and then bucket or re-sync the bucket to avoid overloading the process with items. 

Uploading Files to the Media Library 

• The media library now supports the indexing of all files that support IFilter. For more information about IFilter, see http://en.wikipedia.org/wiki/IFilter. 
• Note :Sitecore does not by default provide search in PDF documents on MSSQL databases, and in neither PDF nor Word documents on Oracle databases.
• In short, IFIlter is a generic interface for indexing documents. Sitecore 7 ships configured to use IFilters to index text in the binary content of media items. To use this feature, you must install IFilters for the types of media items that you want your solution to index. You can use software such as the free IFilter Explorer from Citeknet to investigate the IFilters installed on your system. If the system hosting a Sitecore solution does not have an IFilter for a given media type, Sitecore can only index the metadata stored in that media item, not its binary content. Additionally, whether search results include media items can depend on the encoding of the format of data contained in those media. For example, IFilters may not be able to convert images of text in media items to structured text to parse. 
• Finally, you must install IFilters on the relevant hosts in your production environments (both content management and delivery); having an IFilter installed in a development environment will not allow indexing of that data type in your production environments. 

Bucket Config Tweaks 

• You can tweak your index so that it doesn’t index certain things that you don’t want in the index. This will decrease rebuild time and improve search time. 
• Consider rebuilding your indexes on a computer that has a solid state disk. Incremental updates do not have to be performed on SSD but they will benefit from this as well. If you have one dedicated server that rebuilds indexes and deploys them to an environment, ensure that this server has an SSD. Indexes will not be so big, so a small SSD will suffice — for example 64GB. 
• Don’t shard too many indexes. Sitecore must context switch between these shards and this slows down search time. 
• If you have very large caches, you can see large memory spikes when you run a search. This is normal as a search is filling the ItemCache for the results. Be careful of under-optimized 

Ref : Developer's Guide to Item Buckets and Search