Web Design - Scalability and Speed

Greg Linden reported that every 500ms of latency costs Google 20% of its traffic. Similarly, every 100ms of latency costs Amazon 1% of their sales. The presentation concludes that speed matters (big time), but it is hard to achieve high speeds when you are manipulating large data. Here is where a scalable design matters. My experience with web design is minimal. Maybe you have some good feedback/tutorials/articles on how to achieve scalability when large amounts of data is involved.