In general, I avoid coding contests, especially if they focus on speed. This idea, however, caught my attention as something I'd like to both participate in, and obviously, to see the results. I'm a reasonably proficient Emacs user, and my config is (just checked) just shy of 5k loc mark. I would love to see if all the hours I put into writing that code make any difference.
I'll have more comments later, but the first two, slightly related to each other, would be:
1. Are you sure using just one language for the code is a good idea? It's going to instantly turn off large segments of people who would otherwise like to contribute - no matter which language you choose. Further, "the most popular language" may not even be appropriate: there's probably less Java, C#, and C++ programmers who use Emacs compared to the general population. I think the first step here should be a poll to see which languages are actually popular among Emacsers. Once we have the list, we can prepare a couple of codebases using the top 5 entries (for example.) I think I can contribute here - I don't think there'll be a language I'm unfamiliar with on the list. I'm also reasonably confident that, in case the improbable happened, I'll be able to learn that language in a few days. (Assuming there are docs available and so on, the usual disclaimers :))
2. Another consideration: how much time would participants have to prepare? I believe this is crucial. In my case, I have a bunch of "first-class" lang configurations, for the ones I spent the most time with, and "the rest," with configs just barely above bare-bones. If you give me a month, I guarantee I'll have all the possible tools you could use with the lang prepared, and I'll have quite a few hours of practicing their usage. If I had to go for it right now, and the implementation language happened to be outside of ~10-15 languages I use regularly or used extensively in the past - I don't think I'd be up for it.
> Starts a fresh emacs instance
Why a fresh one? Do you want to measure startup time (and later, performance in general)? But then people on the latest and greatest would have an advantage over someone working on a netbook.
> a emacs lisp script
I wonder if this wouldn't be better handled by an external tool. I don't know which, but it might be worth considering.
You don't need keypresses - commands and functions would be quite enough. To be honest, I'd prefer a short chat with a participant afterward, maybe coupled viewing the replay at 10x the speed. What is valuable, I think, is the thought process behind the chains of commands, and "decompiling" those out of the "assembler" (key presses) of people's minds might prove quite hard.
That's it for now, I'll be happy to talk more about the idea, though I think we should take the conversation off the HN. My email is in my profile :)
Hey, great to see some interest and thanks for the ideas. I am going to keep the conversation going here instead of a private email thread, in case anyone else shows interest in this. I am also thinking about refining the design and asking for more participants on r/emacs + r/spacemacs or even language-specific subs.
I remember checking the thread several hours after posting my comment and assumed that it was so offtopic people didn't engage, hence the delayed reply.
> Are you sure using just one language for the code is a good idea?
The more languages the better, I was only assuming that I would be preparing the codebase myself, hence wanted to maximise ROI by choosing a popular language that I can write.
Adding more languages would be great, however, we would need to keep the code + tests + failures as consistent as possible i.e. no language-specific test failures due to multi-threading unsafety of C++, which Python's GIL prevents. Also I am not sure how useful cross-language comparisons are.
> how much time would participants have to prepare?
Prepare what? Ideally nothing in their language + emacs setup. The whole point is to learn about people's natural emacs workflow and what tricks they use when in the state of flow. I think each participant should only do it once in their "first-class" language, otherwise participants won't enter flow. That way, we control for no previous familiarity with the problem and get the best emacs knowledge and maybe even some elisp snippets others can follow.
To borrow a sports metaphor - I am sure that Tiger Woods can play a decent game of tennis, I would rather get tips from him to improve my golf swing.
> Starts a fresh emacs instance
That was a suggestion born from my workflow that minimises the risk of other buffers getting in the way of the test and preventing users' from leaking personal data during the screen recording.
> commands and functions would be quite enough
Yes and no. Some of Mike Zamansky's youtube videos have an on-screen feed of keypresses, which is interesting to watch for evil-vimlike navigation practices. I cannot promise I will be good at analysing a stream of keypresses and extracting meaningful analyses though, so commands and functions is what I can promise to analyse.
> I'd prefer a short chat with a participant afterward, maybe coupled viewing the replay at 10x the speed.
A chat is a great idea, however, if we manage to get a decent number of participants (ideally 40+ for each language for meaningful stats), my availability for a sync chat will become the bottleneck of the experiment.
I am thinking about replacing it with an async Q&A that the participant can add after completiion.
I'll have more comments later, but the first two, slightly related to each other, would be:
1. Are you sure using just one language for the code is a good idea? It's going to instantly turn off large segments of people who would otherwise like to contribute - no matter which language you choose. Further, "the most popular language" may not even be appropriate: there's probably less Java, C#, and C++ programmers who use Emacs compared to the general population. I think the first step here should be a poll to see which languages are actually popular among Emacsers. Once we have the list, we can prepare a couple of codebases using the top 5 entries (for example.) I think I can contribute here - I don't think there'll be a language I'm unfamiliar with on the list. I'm also reasonably confident that, in case the improbable happened, I'll be able to learn that language in a few days. (Assuming there are docs available and so on, the usual disclaimers :))
2. Another consideration: how much time would participants have to prepare? I believe this is crucial. In my case, I have a bunch of "first-class" lang configurations, for the ones I spent the most time with, and "the rest," with configs just barely above bare-bones. If you give me a month, I guarantee I'll have all the possible tools you could use with the lang prepared, and I'll have quite a few hours of practicing their usage. If I had to go for it right now, and the implementation language happened to be outside of ~10-15 languages I use regularly or used extensively in the past - I don't think I'd be up for it.
> Starts a fresh emacs instance
Why a fresh one? Do you want to measure startup time (and later, performance in general)? But then people on the latest and greatest would have an advantage over someone working on a netbook.
> a emacs lisp script
I wonder if this wouldn't be better handled by an external tool. I don't know which, but it might be worth considering.
> logging keypresses/function calls might feel intrusive
You don't need keypresses - commands and functions would be quite enough. To be honest, I'd prefer a short chat with a participant afterward, maybe coupled viewing the replay at 10x the speed. What is valuable, I think, is the thought process behind the chains of commands, and "decompiling" those out of the "assembler" (key presses) of people's minds might prove quite hard.
That's it for now, I'll be happy to talk more about the idea, though I think we should take the conversation off the HN. My email is in my profile :)