There is no exchange of value between the parties, but there is definitely a cost to peering. You need to physically get your data to the meet-me room, and that isn't free.
Peering is a one-to-one relationship, not "Internet access" like you're thinking. If you peered with Comcast, for example, you'd only be able to reach Comcast customers over that connection. A more "Internet access" type of connection would be transit, which is definitely not free.