I would be curious to know how much when we write, how much of it is self-attention and how much of it is our fore-brain actually trying to make sense? My guess is that the more tired / rushed / burned out you are, the % of self attention increases.
Sometimes watching the news, it seems like 90% of what they say when they are 'vamping' is just self-attention.
Has anyone posted any GPT / Hacker News generated text yet? Wisdom of the crowds, indeed. It'd be interesting to post using it with light editing, especially something that uses upvotes for training.
One of the things I was thinking about was training on your favorite novel, so you could have a sort of conversation with it / ask it questions. A kind of interactive cliff notes. However, as looked into it I realized it was still too much of a markov chain like thing to be functionally useful. Fun idea though.
The real win, in all of this, of course is auto completion in different mediums. Code completion demos are pretty wild - https://tabnine.com/blog/deep/ Come to think about it, you could probably use it for writing academic papers as well assuming you know the content well.
Self-Attention and Human/Computer interaction is a very brave new world. I don't think people really yet know the potential for seismic shift here.
I've trained a Transformer encoder-decoder model (this was slightly before GPT2 came out) to generate HN comments from titles. There is a demo running at https://hncynic.leod.org
It doesn't seem very accurate, there isn't close to enough Electron hate whenever it is in the title.
This is pure gold though:
> How does one make a web app using a standard framework? I've never used it, but it sounds like someone has been able to put together something like a Web app with only one app.
Edit:
This is even better.
> Rewriting a Linux kernel in Rust, by hand, is definitely the right thing to do as a beginner/intermediate programmer.
In my .emacs.d file, the arrow keys, a key with a cursor keys (which are the key bindings for the .emacs.d file above) and then a shortcut to switch to the command that makes use of those.
But I now have a full screen keyboard and mouse.
Here's another way to do it:
M-x { C-c }
You go in the current directory, move up the left arrow key, press escape and hit the backspace key.
> " After some research about the potential implications of being a woman in a Tesla, the first thing Tesla carmaker said was ‘We can do it, but we do the opposite.’”
What does it mean to have an attractive woman in a supercharger with an attractive female face that doesn't have a baby attached?
hncynic 1 minute ago
I think the article needs to be updated to explain what happened here.
As in, Tesla lost a few babies to the first one (the car was still in the hands of two babies) so it was a very minor factor. But what happened to the last one would take a very long time.
The title is a bit misleading. The Tesla was an individual that was given birth in a manner that prevented them from getting it.
They didn't take away the babies from the Model S as well. They took away the babies in the Model S's hands and made it a minor factor, including the fact that the car broke down on the front of the vehicle. The Tesla's only reply is if the Model S would not have had any special features. In my opinion it should have given more minor facts.
Wow. It generated an extremely plausible looking Google Maps URL for me. It doesn't actually go anywhere, but it's crazy to think that the model memorizes random stuff like the common URL parameters and specific formatting of Google Maps URLs. http://maps.google.com/maps?sll=3.00664238,2.2633658&data=!3...
Thanks! I actually planned to make results shareable at the start, but, knowing the internet, I did not like the idea of being held responsible for whatever content (say offensive or even illegal things) people would put into the titles.
Sometimes watching the news, it seems like 90% of what they say when they are 'vamping' is just self-attention.
Has anyone posted any GPT / Hacker News generated text yet? Wisdom of the crowds, indeed. It'd be interesting to post using it with light editing, especially something that uses upvotes for training.
One of the things I was thinking about was training on your favorite novel, so you could have a sort of conversation with it / ask it questions. A kind of interactive cliff notes. However, as looked into it I realized it was still too much of a markov chain like thing to be functionally useful. Fun idea though.
The real win, in all of this, of course is auto completion in different mediums. Code completion demos are pretty wild - https://tabnine.com/blog/deep/ Come to think about it, you could probably use it for writing academic papers as well assuming you know the content well.
Self-Attention and Human/Computer interaction is a very brave new world. I don't think people really yet know the potential for seismic shift here.