Self is mostly important for its implementation and the papers about it, much more so than the language itself. If you're interested in compiler technology, especially the original papers by Urs Hölzle and David Ungar are very approachable. There's a very extensive bibliography here [1]. E.g. the paper on Polymorphic Inline Caches [2] by Urs Hölzle, Craig Chambers, and David Ungar was a seminal paper on optimising method dispatch for dynamic languages (note that lots of other work has been done since, but many of these papers are useful because they're short and simple and give a good foundation for digging into newer techniques)
[1] https://bibliography.selflanguage.org/
[2] https://bibliography.selflanguage.org/pics.html