Google Knows, You Know
Google knows quite a bit about you; you may have already gathered that from general discussions on privacy and Internet habits. If you have an Android mobile device, they likely know where you have been, too. Remember the Apple revelation from a few years ago that the company was keeping GPS data from iPhones? This is a bit like that, but Google is not really hiding it with their location history browser.
If you carry any Google-filled gear (like, say, an Android phone or tablet), there was a prompt during the initial setup that asked if Google could transmit your location data back to the mothership. [The location history browser] is that data. You know how Google Now can auto-magically figure out where you work and warn you about traffic? This is the data that makes that possible (or at least a good chunk of it.)