"Search by image allows you to do a reverse image search and discover all sorts of content that's related to a specific image. For example, search using a picture of your favorite band and see search results that might include similar images, webpages about the band, and even sites that include the same picture," explains Google.
It's interesting to notice that Google uploads the image, instead of using the URL. Maybe Google wanted to make sure that you'll always get some results: not all the URLs are publicly accessible (for example: Gmail's image attachments).
Reverse image search also works for modified versions of an image, so you can use it to find more information about an image, other sites that include the image, the original source of an image. You can also use TinEye, a similar reverse image search engine.