If this really came down to a nondisparagement clause that would be shocking. Getting to make an offer on the rights to some of the best selling music of all time is itself substantial. This explanation would also diminish the emotional/moral attachment Swift still claims to her earlier works -- owning the masters meant a lot to me, but not as much as this tumblr post? The world of celebrity is bizarre so I don't know, but I think there has to be more to it than that. Either that they were never for sale to her, or the price was too high.
Which she wouldn't do, and I wouldn't either: never agree to nondisparagement without substantial consideration.