Decentralization is really the key thing, not whether the work is done by the government versus contractors. The government can be very efficient doing things in-house as long as the efforts are decentralized. The US Attorneys Offices handle tremendous caseloads for their staffing levels relative to private law firms. The Army Corps of Engineers is also quite effective because it's decentralized.

Things that are a mess are often centralized and involve too many contractors. Defense, the VA, etc.