MoE models route each token, in every transformer layer, to a set of specialized feed-forward networks (fully-connected perceptrons, basically), based on a score derived from the token's current representation.
No. Each expert is not separately trained, and while they may store different concepts, they are not meant to be different experts in specific domains. However, there are certain technologies to route requests to different domain expert LLMs or even fine-tuning adapters, such as RouteLLM.
First off, they are basically completely different technologies, so it would be disingenuous to act like it's an apples-to-apples comparison.
But a simple way to see it is that when you pick between multiple large models that have different strengths, you have a larger amount of parameters just to work with (e.g. Deepseek R1 + V3 + Qwen + LLaMA ends up being 2 trillion total parameters to pick from), whereas "picking" the experts in an MoE like has a smaller amount of total different parameters you are working with (e.g. R1 is 671 billion, Qwen is 235).
Many models that use test time compute are MoEs, but test-time compute is generally meant to refer to reasoning about the prompt/problem the model is given, not about reasoning about which model to pick, and I don't think anyone has released an LLM router under that name.
Note that he did not write the paper nor model, nor perform the training. This repo is only downloading a trained model, using a tool to convert it to another format, and using a canned API to plot it and do some inference.
Definitely a high-flying student with a bright future ahead, but this is just stitching together a few API calls.
Basic question - where will all these 747s actually physically go to be retired? Will there be a landfill somewhere with a huge amount of airplane parts? Will they be left to rust in some hangar in the middle of nowhere?
Isn't this only for planes with some value left? Either they're for parts or might fly again? Otherwise, wouldn't you just cut up the airframe and sell it for scrap?
Now, composite planes are a different story. I doubt there's any value in scrap carbon fiber.
For planes that are being retired, they usually take the engines off pretty early because they have some of the most valuable parts for resale in them.
They’d also be drained of all fluids. For planes in storage to be used again later, they fill all the fuel, hydraulic, etc. lines with special liquids to keep them preserved. So it’d be at least a week’s work for a team to get a stored plane going.
The Airlander 10 wrote off their old prototype, and has a maximum payload of 10 tonnes, whereas an unloaded 747 is about 180 tonnes. Remember it's a plane so big that they used to piggyback the space shuttle orbiters (which are not small) on it.
I believe just like storing a car for a long time, fluids are drained and other items removed/wrapped for storage. You can't just jump in the cockpit and taxi out to the runway. They are kept in a state that is relatively easy to restore to functioning status but not in functioning status.
Good luck taking off without fuel and any ground equipment, not to mention any consumables that would have to be replaced after prolonged stay in a desert.
Looks like they leave fuel in the tank if it’s for short term storage, but they drain various fluids if it’s longer term storage. Probably depends on the circumstances.
Precisely—and this circumstance is retirement with no prospect of a quick sale.
At a guess, they’ll probably have limited parts value, only that which would be valuable to operators of 747 freighters. Mainly the engines, if they’re in good nick.
The authentication method is knowing how to fly said plane and getting in said plane in the first place.
The planes that are relatively accessible aren't flightworthy without effort, and the planes that are flightworthy require violating federal law (in the US) to access them.
>and the planes that are flightworthy require violating federal law (in the US) to access them.
That doesn't really answer the question though. Breaking into a house also violates laws, yet it happens all the time. If you've made it onto an airport tarmac, can you just steal a plane?
It's logistically very difficult. Getting fuel into it usually takes a special truck, so you'd need access to that (when a plane is parked overnight it would have a little bit of fuel in it, but not enough to get very far after taking off). Even getting into the plane is difficult - you'd need a stair truck, and somebody to move the stairs out of the way when you're in. If it's parked somewhere where you can't just taxi out, you need somebody in a tug to do a pushback.
So assuming you're that far in, and you know how to start it up, you don't have a flight plan logged, so the tower isn't going to give you clearance to take off. If you take off without permission, they're going to call the air force, so you'd better be in a country that doesn't have a very big air force or a nearby base, and you probably want to choose somewhere where you can get into another country's airspace who isn't friendly with the country you stole the plane from...
it's easier than you might think. for example, on a 747 you don't need external stairs, since access is available via the nose wheel landing gear bay [0] according to a pilot.
That was the sales pitch when most observers thought the 747 was going to be the last subsonic long-haul passenger plane. "Don't worry about the supersonic plane we're currently designing, you'll be able to sell your 747 as a great easy-conversion cargo plane once the SST comes out."
Google "desert aircraft boneyards". They will be parted out for pieces to keep remaining models of the same type flying, and when that's not economical anymore, they are cut up for aluminum scrap.
That's not too surprising, tbh. Building the baseline solution or platform from the ground up is often the most difficult part, and by the end of such a project, you end up with a 100 ideas about how to make it better. But either due to time constraints, a "perfect is the enemy of done" philosophy, or simply the fact that you reach a natural crossroads and there are multiple valid ways forwards, you have to draw a line in the sand and release at some point. It's natural that there my be low-hanging fruit for some people to pick up and improve on for some projects, particularly if they have different end goals than the initial project had.
Another anecdote to illustrate that - he doesn't truly legally have to ask permission to do these parodies, but out of courtesy and respect, he always does. And he'll usually refrain from making a parody if a song if the artist has a genuine reason why (e.g. Paul McCartney didn't have an issue with him parodying "Live and Let Die", but Weird Al was trying to turn it into "Chicken Pot Pie" and McCartney, a staunch vegetarian, didn't love that idea).
To parody without a license you can only take the minimum amount of the source to bring to mind the original. See, e.g., Campbell v. Acuff-Rose Music, Inc., 510 U.S. 569 (1994). So, Yankovic probably does need a license from the owner of the composition, because his versions usually borrow the entire source song and don’t change the tune or arrangement. Also, many of Yankovic's songs aren't parodies but take a well-known song and take the lyrics in a completely different direction.
As I understand it, he does also seek permission of the original artist, who isn’t always the owner of the copyright, as a professional courtesy.
> you can only take the minimum amount of the source to bring to mind the original.
That's not what the ruling you refer to says, and it's not what the statute says. The ruling's text is at https://www.law.cornell.edu/supct/html/92-1292.ZS.html and mentions nothing along the lines of "minimum". The Supreme Court held in that particular case that the portion of the lyrics copied was not excessive, and left it up to the lower court to decide whether repeating the bass riff was excessive.
The statute's general guidelines are that all four of the following factors must be taken into account when determining fair use:
(1) the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes;
(2) the nature of the copyrighted work;
(3) the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and
(4) the effect of the use upon the potential market for or value of the copyrighted work.
Factor (3) is the only one that deals with how much of the source material can be used, and it also does not support your assertion. And those rules aren't just for parodies.
You can just play the actual song the other person wrote. You’ve heard of the concept of covers right?
I don’t need anyone’s permission to record my own version of any song, and I can change the words or otherwise rearrange it however I want.
I might owe them publishing royalties but that’s an entirely different conversation.
The article implies it’s a grey area but that’s not actually true, it’s not really borderline at all, at least not for creating the songs. Al could have completely ignored the artists opinion he just chose not to.
With that said, the issue of a sync license could be a little more complex, and maybe is what the article is referring to. That could have some impact on the ability to create videos, though that would generally seem to hold up well to a fair use defense.
> I don’t need anyone’s permission to record my own version of any song, and I can change the words or otherwise rearrange it however I want.
You can't change the words - doing so makes your cover a derivative work, and not covered by the usual compulsory licence. You can change the arrangement
You’re right and I should have been more clear. Yankovic does need a license, but not permission because the compulsory license scheme for “covers” would suffice.
True story: a group of Indian scientists with sense criticize a right-wing group of pseudo-scientists making outlandish claims.
Impression left by BBC headline: Indian scientists are wack
Like, come on. One could do the same by compiling the views of every ridiculous theory espoused by people like Ben "grain silos" Carson or young earth creationists in America and write "American scientists dismiss reason". Are there a significant amount of people that believe nonsense like this? Sure, maybe. Is that representative of American scientists as a whole? Of course not.
And it's the same thing here. I guarantee you a biology grad student at some place like IIT is under no delusions about the origins of stem cell research.
reply