1) Enable caching!!!
http://www.joomlahacks.com/content/view/84/64/
http://extensions.joomla.org/extensions/1341/details
2) Limit the number of articles and content blocks displaying on your front page
3) In your Admin section, enable caching to reduce the number of PHP/MySQL calls
4) In your Admin section > Global Configuration, enable debugging and then check how many MySQL queries are being generated. This should help you pinpoint any plugins on your site that are occupying excess resources or causing your site to load slowly
5) Disable any modules, componenents, or mambots that you are not using, as well as any plugins that may be troublesome. Generally, plugins that collect data from external sites (like Digg, social networking sites, game servers, etc) tend to be attributed to site interruptions
6) Try installing the SM2 optimization plugins, which you can get here
7) Run your site against the online optimization tool to find out what parts of your site are taking the longest to load
8) Try using a robots.txt (robotstxt.org) file to limit search engine activity on your site to only the parts that you want to be indexed in search engine.