I am assuming that if something is technically possible, they're doing it. So, they're tracking telecom metadata and storing content because they can.
In fact I believe that the only reason our physical mail was mostly not read had nothing to do with the 4th Amendment, and everything to do with that it was logistically impossible and still have a working postal service, and difficult to keep secret.
With modern mail sorting machinery, I'm sure there is metadata being collected on physical mail.
>So, they're tracking telecom metadata and storing content because they can.
Correct me if I'm wrong, but to theoretically record every single phone conversation in America would require unbelievable amounts of data storage, would it not? Even if it's stored in a highly compressed format.
The metadata I can believe, but I find it kind of unlikely they have the means to store the contents of every call.
"Brewster Kahle, a computer engineer who founded the Internet Archive, has vast experience storing large amounts of data. He created a spreadsheet this week estimating that the cost to store all domestic phone calls a year in cloud storage for data-mining purposes would be about $27 million per year, not counting the cost of extra security for a top-secret program and security clearances for the people involved."
Storage is cheap, and only getting cheaper. You could archive things older than a certain date onto cheaper media. Or consider converting audio into text-transcripts after a period of time, and you can dramatically reduce your storage requirements. Either way, $27 million isn't really much.
Others on HN have done the math on this. It's doable, at bad but reasonable quality, with 2% of the storage capacity of NSA's new datacenter corresponding to roughly one year of all American voice traffic.
> Correct me if I'm wrong, but to theoretically record every single phone conversation in America would require unbelievable amounts of data storage, would it not?
GSM voice codec is 9600bps = 1200 Bytes/s
x60 sec, x20 min per day, x365 days per year = about 500MB per person per year.
times however many people there are in the US, that's not a whole lot of storage and definitely a mere fraction of the NSA's data-centres.
Whatever they're technically capable of doing, I assume they're doing it. If they can't store everything yet, I assume they're working on it. I also assume they're somewhat ahead of whatever we commonly think is possible.
Maybe not everything, maybe not now, but close to it and soon.
According to this article (https://sumanrs.wordpress.com/2012/04/14/youtube-yearly-cost...), about 76 petabytes is stored to Youtube per year. (calculated from the released info, that "one hour of video is uploaded to YouTube every second."). So Petabytes/day is probably doable for them.
In fact I believe that the only reason our physical mail was mostly not read had nothing to do with the 4th Amendment, and everything to do with that it was logistically impossible and still have a working postal service, and difficult to keep secret.
With modern mail sorting machinery, I'm sure there is metadata being collected on physical mail.