I believe the concept is a little different than what you stated. It starts with the assumption that the Universe is actually in or very close to a maximal entropy state, which is orders and orders of magnitudes more likely than the low entropy state we perceive. The idea of a Boltzmann brain is that a downward fluctuation in entropy from this maximum entropy state could result in forming a consciousness with our associated memories and experiences and this is more likely, statistically speaking, than the alternative possibility: that we actually live in an extremely low entropy state of the Universe.