Signal doesn't store anything about your account on the server except last login time and when you registered. It doesn't store a contacts list, so it used your own, assuming you granted it access to do so.

Contrast to MSN, which kept your contacts on the server, as well as information about your account, groups, your plaintext messages, etc.