Plastic water bottles are a HUGE source of waste. For one many of the bottles never make it to a recycling center and end up in a landfill where it will take FOREVER for them to decay. Recycling is better but still not as good as just plain not buying them! Instead invest in one reusable water bottle, like a Nalgene bottle. They won't retain odors or make your water taste bad, you can even throw in some ice cubes to keep the water cold!
Here are some good sites to get you going!