Even if we were not past a hard takeoff point where AIs could decide for themselves what to work on, the things that would be created in all areas would be incredible.

Consider every time you played a game and thought it would be better if it had x,y, or z. Or you wished an application had this one simple nrw feature.

All those things would be possible to make. A lot of people will discover why their idea was a bad idea. Some will discover their idea was great, some will erroneously think their bad idea is great.

We will be inundated with the creation of those good and bad ideas. Some people will have ideas on how to manage that flood of new creations, and create tools to help out, some of those tools will be good and some of them will be bad, there will be a period of churn where finding the good and ignoring the bad is difficult, a badly made curator might make bad ideas linger.

That's just in the domain of games and applications. If AI could manage that level of complexity, you can ask it to develop and test just about any software idea you have.

I barely go a day without thinking of something that I could spend months of development time on.

Some idle thoughts that such a model could develop and test.

Can you make a transformer that instead of linear space V modifiers it instead used geodesics? Is it better? Would it better support scalable V values?

Can you train a model to identify which layer is the likely next layer purely based upon the input given to that layer? If it only occasionally gets it wrong does the model perform better if you give the input to the layer that the predictor thought was the next layer. Can you induce looping/skipping layers this way?

If you train a model with the layers in a round robin ordering on every input, do the layers regress to a mean generic layer form, or do they develop into a general information improver that works purely by the context of the input.

What if you did every layer on a round robin twice, so that every layer was guaranteed to be followed by any of the other layers at least once?

Given you can quadruple the parameters of a model without changing it's behavour using the Wn + Randomn, Wn - Randomn trick, can you distill a model To .25 size then quadruple to make a model to retain the size but takes further learning better, broadening parameter use.

Can any of these ideas be combined with the ones above?

Imagine instead of having these idle ideas, you could direct an AI to implement them and report back to you the results.

Even if 99.99% of the ideas are failures, there could be massive advances from the fraction that remains.