This. What mattered from the standpoint of US decision making wasn't what the Germans were actually doing, but rather what the US leadership thought the Germans were doing.
Or more specifically, from an intelligence viewpoint, what the Germans could be doing. Intent was well established when Hitler declared war on us (big mistake), but before that always comes capabilities. They had the uranium (something that Japan never had), they had the industrial capability for the plutonium path (ditto), and they most certainly had the scientists (even after chasing all their Jews away). If they'd gotten serious about it sufficiently earlier than we did....
