Websites are incredibly interested in what their visitors do and where they go. The information lets the site know where the person came from, how often they visit and what pages they look at. Dubiously, cookies can also be used to track sites a person is visiting after leaving the site issuing the cookie.
A cookie is a rather simply piece of code that is placed on your server. How does it get there? When you visit a site, a particular procedure is followed from a technical point of view. When you click a link to the site or type in the domain, you make a request upon the server hosting the site. The server then sends you the file, a web page, which corresponds to the request you have made. The cookie is contained in the response and sits on your hard drive.
Cookies often get far more criticism than they deserve. In a vast majority of cases, the site issuing the cookie is merely trying to track data so it can manipulate its site and advertising. This helps visitors by giving them more pages they are interested in and eliminating uninteresting pages.
Cookies can often make life convenient as well. If you return to a site frequently, the cookie will often allow you to get into password protected areas without going through the login process. Admittedly, this is a minor benefit.
If the idea of cookies on your computer is offensive, you can remove most of them by taking minor steps. If you are using the Explorer browser, just click the "tools" button. Scroll down to the bottom of the menu and click "Internet options." A popup should appear with a button in the middle of it with the words "delete cookies". Click it and you should be cookie free.