Simple answer: it delivers news about a site to you.
Think of newspapers. When you subscribe to RSS feeds it’s like subscribing to newspapers.
The only difference is, (1) it’s digital and (2) you need some sort of program to receive the content.
These programs range from web services (My Yahoo! or Google Reader), to broswers with built-in feed management (Firefox or Opera), and even separate programs just for such use (Newsplorer). For a full list of newsreaders visit Wikipedia’s Entry.
My feeds comes in two flavors: