Many people also don't realize how much of the current lakefront is built on fill - Grant Park is largely built atop fill from debris from the Chicago Fire that was dumped into the Lake.
If you'd like to take home a piece as a souvenir, head over to 12th Street Beach (immediately south of the Adler Planetarium). The stuff piles up on the south end of the beach every time there is a storm on the lake. Lots of pieces of bricks, terra cotta, porcelain, etc. Tons and tons of beautiful colored glass that has been tumbled smooth by the waves. Go in the morning and you'll see treasure hunters picking out the best stuff. Rumor has it there every once in a few years someone finds a diamond - it is definitely true that the Great Lakes have plenty of diamonds from northern Canada that were deposited by glaciers in the last ice age.