I practice kata every day. I typically show up to work 20 - 30 minutes early and pick a problem to work on. I record my answer for later reflection. Sometimes I will repeat a solution I came to before, sometimes I will refine an old approach, and other times I will think of a new way to approach it. I keep all of my problems that I collect and the solutions I come up with in a git repository.
Sometimes I will call to mind an interesting solution I came up with. Having the repository is handy so that I can pull up the problem, the solution I am interested in and perhaps diff it against a previous, similar solution I had came up with before. I use this time to reflect on the changes I made, how it changes the characteristics of the algorithm (time, space, bounds, etc), and whether it states the intent with more clarity than before. I keep these reflections in a journal.
I know I've improved when I notice in my journals that my language changes dramatically from dense, technical language to simple, straight-forward language without losing any of the accumulated knowledge.
>I know I've improved when I notice in my journals that my language changes...
While what you're doing is admirable--I wish more developers did this--the central question is not whether you improved, but whether you improved more than if you'd done other work. For example, learning a new language, refactoring, etc.
I end up, "refactoring," by consequence of repetition I suppose. There are some days when I can't think of another way to write a fibonacci function (a trivial example... without including obfuscated methods, there are only so many effective ways to implement a function that computes the n-th number of the sequence). On those days I may end up calling to mind a previous solution and simply, "refactor it."
I have conflicting feelings about learning new languages. I do it every now and again but for a craftsperson I don't feel that it is the ideal approach. As some monks I've heard say so it is with programming: you can be fluent in many languages but master none. Real mastery requires dedication and focus over time. But that's not the whole story, is it?
I've met too many developers who will put C on their resume under the assumption that they are capable of writing C programs. More often than not it's a bold-faced lie, but when it isn't I find they are certainly capable of writing a C program and getting something to compile. However there are very few who understand what sequence points are, that still treat the linker like black magic, and don't fully grasp how array and array pointer parameters are changed by the compiler.
The point I'm getting at is that C is generally one of many languages on a programmer's resume. And it's usually an after-thought. However if you really grill someone on the soup-to-nuts of just one of these languages you can find holes in their knowledge. They learn enough to get something useful done but they don't know enough to do truly masterful things with the language. I think about it like being a musician: you can learn enough about an instrument to join a jam but to push the boundaries you need to master one. And mastery is something I find thats consistent, dedicated practice.
As to how to measure whether I could have improved more if I had done something else other than master one thing... I think there's a lack of bounds there that should be determined by your goals. If your goal is to join a jam and play along then you want to learn as little as possible up front and as much as you can as you go along. I think this is the par for modern programmers these days. But if your goal is to push what's possible with a particular instrument then generalizing your knowledge is not nearly as effective as studying music with that instrument alone. Both are valid approaches and it's a matter of limiting the scope of possibilities based on what you want to achieve.
Though mastery tends towards an asymptotic function reaching, but never achieving, perfection.
Sometimes I will call to mind an interesting solution I came up with. Having the repository is handy so that I can pull up the problem, the solution I am interested in and perhaps diff it against a previous, similar solution I had came up with before. I use this time to reflect on the changes I made, how it changes the characteristics of the algorithm (time, space, bounds, etc), and whether it states the intent with more clarity than before. I keep these reflections in a journal.
I know I've improved when I notice in my journals that my language changes dramatically from dense, technical language to simple, straight-forward language without losing any of the accumulated knowledge.