I wouldn't be so worried about LLMs making guns. The SketchIT project found that it's very difficult to describe mechanical device to other humans using just text, images are needed too[0]. I'd also worry about the gun produced being structurally sound, how can one be so sure that the gun barrel hasn't been hallucinated to be too thin?
Guns and other mechanical devices don't exist alone. A gun must interface with a bullet, a part of an aircraft must interface with other parts. So CAD AI must be able to understand the geometric context of the parts it is making.
That being said, I think AI will soon be capable of making mechanical devices. There has been some improvement in physical reasoning benchmarks like PHYRE[1]. Understanding physical reasoning and how multiple objects move with respect to each other is important in the synthesis of new mechanical devices.
SketchIt[0] demonstrated that by making reduced 2DoF description of how pairs of objects in a device may move with respect to each other, it's possible to synthesize a new device which performs the same function.
Solving PHYRE problems requires reasoning with larger degrees of freedom. The first example on the homepage has something like 5 objects which each have 3 positional DoF (translation and rotation). Even reasoning with 3DoF is quite difficult for approaches like those used in SketchIT.
Given that approaches like slotformer[2] already do somewhat well at solving these huge DoF problems, I don't think we're very far from AI being able to design complicated mechanical devices.
Guns and other mechanical devices don't exist alone. A gun must interface with a bullet, a part of an aircraft must interface with other parts. So CAD AI must be able to understand the geometric context of the parts it is making.
That being said, I think AI will soon be capable of making mechanical devices. There has been some improvement in physical reasoning benchmarks like PHYRE[1]. Understanding physical reasoning and how multiple objects move with respect to each other is important in the synthesis of new mechanical devices.
SketchIt[0] demonstrated that by making reduced 2DoF description of how pairs of objects in a device may move with respect to each other, it's possible to synthesize a new device which performs the same function.
Solving PHYRE problems requires reasoning with larger degrees of freedom. The first example on the homepage has something like 5 objects which each have 3 positional DoF (translation and rotation). Even reasoning with 3DoF is quite difficult for approaches like those used in SketchIT.
Given that approaches like slotformer[2] already do somewhat well at solving these huge DoF problems, I don't think we're very far from AI being able to design complicated mechanical devices.
[0]https://dspace.mit.edu/bitstream/handle/1721.1/6773/AITR-157...
[1]https://phyre.ai
[2]https://slotformer.github.io