That's exactly how I read it. It was the first obstacle to overcome. "OK, they're walking different speeds, and the statement means that neither of them changed their own unique speed over the course of their walk."

It took a bit of convincing myself, but eventually I became confident that it did not mean that they were each walking at the same rate as each other after their lunch meeting.

But even then, I struggled with figuring out how I arrived at 6AM.

I like the method of switching to the 24 hour clock (as opposed to the AM/PM clock), and setting their arrival times as 16:00 and 21:00, with their meeting at 1200, then solving it.