The set of indexes are a form of data structure though, with all the state juggling and heuristics that usually go along with them (What access pattern requires this index? When do I have too many of them? What locks when I add/drop an index?). This adds complexity and unpredictability to the user.
I would argue that dropping and creating indexes is much less work than refactoring large parts of your codebase just to change a data structure.
Also, you can have multiple indexes on a table and the RDBMS ensures that they are always synchronized, which is something you won't achieve if you code the data structures yourself.
