I haven't more than skimmed the first and last page, but if you want to really understand this issue you have to go back at least to the '70s, particularly:
The Church Commission.
The purge of James Jesus Angleton, and more importantly the effective end of the agency's counterintelligence function, which is very much a part of this particular screwup; it's never a popular thing, no one likes being told their informant is a double, but ignoring the possibility is lethally expensive. (See also Aldrich Ames, who's lifestyle should have been caught by counterintelligence as another example of a failure in this area.)
Jimmy Carter's DCI, Stansfield Turner, and e.g. how he riffed 800+ HUMINT operators in just one purge.
The more I learn about the history of intelligence services post WWII in America, the more as a taxpayer it ticks me off. Not the agency trying to do loopy things like poison Castro's underwear, but the wholesale ripping apart of decades of work just because the politics change one way or another. Want to fund overthrows? I don't like it, but fine, if it's done according to process. Want to abolish funding overthrows? Fine, if it's done according to process. But tearing down entire structures? Effectively destroying all HUMINT in the 1970s? It's insanity. Completely crazy. The intelligence agencies never managed to stay above politics, which is a shame.
The problem is not that the intelligence agencies can't stay above politics, but that the politicians meddle with the intelligence agencies for political reasons. Despite what some conspiracy theorists believe, the US intelligence agencies are run by the elected officials, not the other way around. This is good for a lot of reasons, but does leave them subject to the whims of politicians.
Yes to what both of you are saying. But it's not quite just "politics", I would say. Perhaps look at this as an evolution of the post-WWI principle that "Gentlemen don't read each other's mail"? (http://en.wikipedia.org/wiki/Office_of_Strategic_Services#Or...)
(Me, after the WSJ recommended it I just got a copy of The Deceivers by Thaddeus Holt, 1148 pages (!!!) on "Allied Military Deception in the Second World War". Very interesting.)
I think there's another problem. Arab intelligence services tend to be brutal, so I'm not surprised that _they_ thought that their course of action was a good idea, but was "let's torture this Islamic-fundamentalist blogger into infiltrating al-Qaida for us" really a course of action that raised _no_ moral or common-sense alarm bells at the CIA?
And where do I go to get my money back? I would rather be killed by a suicide bomber than defended by torturers. (I say this as a conservative -- and as someone who believes in objective right and wrong.)
Why are you "fine" with US covert/clandestine operations as long as they're done "according to process"?
I contend our foolish interference overseas lead directly to the blowback of 9/11 and current tensions with Iran, for starters. Nothing good has come to US citizens from the nonsense the intelligence community gets up to. It should be cut down to a twentieth the size ($25 billion or whatever it is these days is ridiculous).
Because I am fine with the government doing things I do not agree with -- this is why we have elections. The point isn't to get into some kind of spitting match over who did what when. Let's stipulate that duly elected officials acting within the constitution instructed the CIA to do things we do not like.
To respond to the fact we don't like it, we elect new officials. We do not terminate the entire national intelligence program. After all, the next bunch of guys in Washington may need a CIA due to no fault of our own, and it take decades to spin one up. Trashing it in a moment of political anger takes away options we might need 20 years from now.
Except it's never as long as "20 years from now". Carter's desire for regime change in Iran certainly didn't take into account any reports on possible downsides (including external ones, like Sadam deciding Iran was vulnerable and starting a war that killed a million men), and he dearly needed serious HUMINT after the hostages were taken (which was probably #2 after the economy in ensuring his defeat in 1980).
Does anyone remember how totally surprised he was by the Soviet invasion of Afghanistan? We'll never know if the CIA could have helped avoid that blindness (probably not, too much of it was willful or just not understanding simple realities like the nature of the Soviet Union and its leaders), but by deliberately gutting its HUMINT in favor of "national technical means" (SIGINT and TECHINT (e.g. spy satellites)) he all but ensured he'd be flying blind afterwords.
In all fairness, just to take the other side, there has been a push for a long time to replace people with machines, both in intelligence and the military. Satellites don't need the constant care and feeding that people do, and pictures don't lie (supposedly)
Looking back, of course, it's been blindingly obvious that there are situations which people are critical -- counter-insurgency, to use a military example. But still technology has always been seductive as a "clean" way to get things like intelligence done.
If I remember correctly, when we axed so much of HUMINT, one of the justifications was that satellites could do most anything a person could do. Totally crazy, of course.
While you certainly have a point, one that might be particularly attractive to one of Rickover's nuclear engineers, I'd counter by pointing out the long and storied history of "spies" and how they made critical differences in war throughout the ages up into the Cold War in the '60s (at least).
On the other hand, ULTRA was officially declassified in 1974 and in 1977 one might still have the conceit that the NSA could break ComBlock ciphers.
All in all, though, I would suspect that the greater motivation was neutering the icky parts of the CIA that the Democratic party turned on post-1968. Insane on its face "satellites could do most anything a person could do" sounds more like a justification and/or rationalization.
For what? What have they EVER gotten right? This utterly failed 60 year experiment needs to be shut down. The actual intelligence functions should be absorbed into the pentagon with a much smaller budget.
Military intelligence is inevitably oriented towards the needs of the military and in the US the needs of the particular branches who are calling the shots at any one point in time.
Our national political leadership needs higher level political intelligence, like "what's this foreign leadership likely to do?" and so on, in addition to what they can do (based on e.g. their military resources).
There's also a selection bias in your comment "What have they EVER gotten right?" We hear all about many of their mistakes, but by definition we don't hear about most successes until they're declassified many decades later.
> leadership needs higher level political intelligence
Subscribe to Foreign Affairs. The idea that you need a multibillion dollar agency to sound out the geopolitical landscape in today's world is absurd.
Let's be honest that their primary function is global agitprop and covert ops. Analysis is a bolt-on, and not better than what you can get in the private sector.
You're leaving out the storied concept of spies, double agents, etc., who can provide unique windows into a foreign country. Plus help the propagation of disinformation (e.g. it's a lot more effective when you know what your enemy fears). If there's a place for them in our national intelligence infrastructure, where should that be?
On the other hand I'll certainly agree that CIA analysis is poor at best, and note that it's particularly subject to politics (e.g. see the 2007 "Iran stopped their nuclear weapons program" analysis (not sure if it was from the CIA but the point holds)).
I'd close with noting that covert ops have their place. E.g. in denying the Soviets effective natural gas compressors (and therefore foreign exchange) for their pipelines to Western Europe. Technology embargoes forced them to roll their own, and we fed them bogus info which resulted in very poor results (as I recall a lot of failures and only being able to ship 1/2 the gas they'd intended).
My first question is where you do decide to strike the balance: in counterintelligence, do you prefer false positives or false negatives?
I haven't tried to get a full measure of him and his actions (for one thing, I suspect that's going to be impossible until everything from the relevant periods are declassified), but my major point is that the counterintelligence function is essential and we pretty much just don't do it.
The Church Commission.
The purge of James Jesus Angleton, and more importantly the effective end of the agency's counterintelligence function, which is very much a part of this particular screwup; it's never a popular thing, no one likes being told their informant is a double, but ignoring the possibility is lethally expensive. (See also Aldrich Ames, who's lifestyle should have been caught by counterintelligence as another example of a failure in this area.)
Jimmy Carter's DCI, Stansfield Turner, and e.g. how he riffed 800+ HUMINT operators in just one purge.