Hacker Newsnew | past | comments | ask | show | jobs | submit | wz1000's commentslogin

> Thank you UN for not being able to do your job!

The UN is simply a forum for countries to meet and discuss international affairs. It is not a global governing body, and wields no power not authorized by its constituent member nations.


This is an important distinction and to understand why, it's helpful to read about its predecessor, the League of Nations which due to mismanagement failed spectacularly in its goals to prevent conflicts between nation states.


From their PR discourse, they don't seem like just a forum for countries to meet and talk.


In The Expanse (great sci-fi novels that just got a new TV season last week), the UN becomes the governing body of Earth after some climate-related disasters that individual countries could not or would not handle (I think).


What does it have to do with reality, though?


> I don't disagree with the concept of a "bug bounty for information" but I'm struggling to come up with an idea of what that would look like in a way that is either more effective or different than patents.

https://en.wikipedia.org/wiki/Federal_grants_in_the_United_S...


That's exactly the example I had in mind when I wrote the sentence you quoted.

Pharma grants have their own issues. Our current mixed system with both private investment and public investment allows us to leverage the strengths and weaknesses of both allocation schemes.


I feel like, ideally, such a system of incentivizing novel ideas and information should be decentralized. Bug bounties, for example, is a decentralized form of incentivizing public works.


Zathura supports epubs using zathura-pdf-mupdf


Interesting you pick on any mention of Foucault as giving away ideological leanings, since, incidentally, Graeber was not a big fan.

From "Fragments of an Anarchist Anthropology":

> Academics love Michel Foucault’s argument that identifies knowledge and power, and insists that brute force is no longer a major factor in social control. They love it because it flatters them: the perfect formula for people who like to think of themselves as political radicals even though all they do is write essays likely to be read by a few dozen other people in an institutional environment. Of course, if any of these academics were to walk into their university library to consult some volume of Foucault without having remembered to bring a valid ID, and decided to enter the stacks anyway,they would soon discover that brute force is really not so far away as they like to imagine—a man with a big stick, trained in exactly how hard to hit people with it, would rapidly appear to eject them.

http://abahlali.org/files/Graeber.pdf

And "Dead Zones of the Imagination":

> Consider the hegemonic role, in US social theory, of Max Weber in the 1950s and 1960s, and of Michel Foucault since the 1970s. Their popularity, no doubt, had much to do with the ease with which each could be adopted as a kind of anti-Marx, their theories put forth (usually in crudely simplified form) to argue that power is not simply or primarily a matter of the control of production but rather a pervasive, multifaceted, and unavoidable feature of any social life ... Foucault was more subversive, but in a way that made bureaucratic power more effective, not less. In his work on asylums, clinics, prisons, and the rest, bodies, subjects—even truth itself—all become the products of administrative discourses. Through concepts like governmentality and biopower, state bureaucracies end up shaping the parameters of human existence in ways far more intimate than anything Weber might have imagined.

> Foucault’s ascendancy, in turn, was precisely within those fields of academic endeavor that both became the haven for former radicals, and were almost completely divorced from any access to political power—or, increasingly, from any influence on social movements as well. This gave Foucault’s emphasis on the “power/knowledge” nexus—the assertion that forms of knowledge are always also forms of social power, indeed, the most important forms of social power—a particular appeal.

https://www.journals.uchicago.edu/doi/pdfplus/10.14318/hau2....


Thanks for the references.

At some point between 2006 and 2015, he must have decided that Foucalt was accurate enough on the subject of exercising power in society, to quote him so unquestioningly in this article.

Unless, of course, there's a distinction between the subset of "power" that this article deals with, and those he previously criticised Foucalt on, that make all four passages congruent with each other.


Very ironic mentioning semiconductors, when it was the protectionist US-Japan Semiconductor Trade Agreement pushed by Reagan that propped up the American semiconductor industry, while leading to the decline of the Japanese industry


Thank you for drawing my attention to this slice of US foreign policy history w.r.t. to high-tech.

“At the insistence of the United States, in 1986 Japan agreed to limit its exports of semiconductors, mainly the "dynamic random access memory" (DRAM) chips, to America. These chips are used in high-tech consumer electronics equipment like computers and video cassette recorders. The agreement expires this July 1, and the Bush Administration thus soon must decide whether to renew it. Doing so would make Washington a hypocrite in its free trade efforts to open markets abroad for American products. The 1986 chip agreement, after all, restricts trade, ostensibly to help some American segments of the semiconductor industry. The agreement in fact has harmed American computer manufacturers, who have found themselves paying higher prices for computer chips. This makes American computer manufacturers less competitive and drives up computer prices for all Americans.” [by Bryan Johnson, published January 24, 1991 – 20 min read]

!! @ “high-tech consumer electronics equipment like […] video cassette recorders”

https://www.heritage.org/asia/report/the-us-japan-semiconduc...

It would be an interesting couple of evening's research to compare and contrast the competitive rise of Japan in the 80's and the rise of China in the 10's and how the US reacted to both and how both reacted back.


The trade war with Japan in the 80's really helped Samsung jumpstart their DRAM business -- South Korean companies, Samsung and SK Hynix, account for 80% of global DRAM production today.


Here is what Iranians see from major American presidential candidates considered to be "highly respected on both sides of the aisle": https://www.youtube.com/watch?v=o-zoPgv_nYg


> But what do you expect to happen when you forcibly take all of the British resources?

Very strange how oil underneath the Iranian soil can be considered a "British resource".


It is a fair statement because ownership was granted via concession to the British, originally by the Qajar dynasty. Moreover, a company is allowed to own resources and infrastructure outside of its country of domicile, just the same as Royal Dutch Shell is allowed to own resources and infrastructure in the USA.

https://en.wikipedia.org/wiki/D%27Arcy_Concession

https://en.wikipedia.org/wiki/Anglo-Persian_Oil_Company

I often see on HN that, when the topic of Iran comes up, people who don't know much about the history of the country try to interpret its facts through a modern lens that is tinged with a hint of leftism. The reality is that the development of the global petroleum market was a fiercely capitalistic and competitive phenomenon that yielded immense wealth for major oil companies and host governments alike. That context cannot be amputated when discussing topics like Mosaddegh and the British sphere of influence in Iran during the first half of the 20th century.


Why should a concession made by a monarch pre-constitutional revolution be expected to be upheld by a democratically elected leader operating after significant changes in the form and nature of government?

I have noticed all such principled defenses of "property rights" and "capitalism" are built upon a very specific and convenient view of what exactly counts as "legitimate property". Was the oil even the Shah's to give away in the first place?


Why should a treaty signed by Obama be expected to be upheld by Trump?

Continuity of governance, honoring good-faith agreements, maintaining confidence of foreign investors...the list goes on.

Are you suggesting that all governmental responsibilities ought to be thrown out the window every time power changes hands? This is not a realistic perspective in most of the third world, where power changes hands frequently and change of government is often established via change of regime.

Ultimately Iran made a killing on the oil business, and that would never have happened without petroleum concessions. In the parlance of the oil business, a "concession" is a deal where the host government allows oil rights in return for profit share, ownership, or some other financial benefit. It is an asymmetrical but symbiotic contract, and the term doesn't carry the same pejorative connotation as it would in standard parlance.

Not to mention that the monarch in question was also the one who ratified the Persian Constitution of 1906.


I think that people would question whether the agreement was in fact good-faith. The British company had agreed to various obligations to improve the working conditions of Iranian workers, train more Iranians to work in administrative roles, and generally contribute to the development of infrastructure in the country. It had done none of those things in the decades since signing the deal.

Under that light, it's not really a case of buyer's remorse, where Iran signed a fair deal, and unjustly decided to renege on it. It's a case of the British company acting exploitatively, violating their agreement, refusing to renegotiate, and the Iranians declaring the agreement void as a result.


Are we talking about the D'Arcy Concession? I ask because none of what you are describing is found in the text of that agreement.

It's immediately obvious upon observation of modern Iran's highly developed petroleum industry that concessions sparked the development of infrastructure on a massive scale.

> Under that light, it's not really a case of buyer's remorse, where Iran signed a fair deal, and unjustly decided to renege on it.

I think it's precisely a case of buyer's remorse. In Iran prior to the development of infrastructure, 16% share in profits generated by AIOC with no initial capital commitment from the Qajars sounds like a great deal. In Iran after the wells are pumping, 16% share in profits sounds like a pittance. I reject your framing of the issue as an exploitation by British interests. That may be a palatable narrative for the times we live in today, but it's a distorted perception of the actual facts. Petroleum production was the single greatest driver of wealth and development in 20th century Iran. If not for foreign investment, the industry could not have boomed as it did.

D'Arcy took massive personal risk and even took the British government as an investor, but he failed several times in the process of exploring for oil before he was ultimately successful. Risk is the nature of the wildcatting business.

If you study the process of oil rights and nationalization in the Middle East, you will see that it is a topic marked by extremely brutish behavior from local parties as well as faraway beneficiaries like the US and the UK. Even OPEC is full of deception about production numbers as it sets production amongst member nations.

For that reason as well as others beyond the scope of the current conversation, there aren't many clean hands in the oil production business. Look at the massive corruption and cronyism taking place in Venezuela's PDVSA. Iran has the same kind of problem, where the country's dictators finance their corrupt practices using petroleum income.


I'm talking about the 1933 renegotiation. However, the original D'Arcy Concession terms were also regularly violated by the British.

This document from 1952 has an excellent summary: http://documents1.worldbank.org/curated/en/74019146804407210...

See Section II (D) beginning on page 4, for violations of the original concession, and Section III (D) beginning on page 16 for violations of the 1933 agreement.

I'm not simply casting the British behavior as exploitative in some hand-waving appeal to the evils of imperialism. I'm talking about real, substantive, and specific violations of their own agreement.


That document highlights Iran's government's objections to AIOC, and we have to contextualize those objections within the backdrop that Iran had already nationalized AIOC a year prior, and needed to justify its actions internationally in order to keep selling oil abroad. World Bank member nations and other buyers also needed to know they would not invite grievances with the UK by continuing to buy from a nationalized company.

The author of the document you cite says that prior to the 1933 agreement, AIOC withheld royalties under guise of covering damages but really to squeeze Iran into accepting the new agreement; the problem with this logic is that damage to a pipeline also causes revenue loss, so you can't indemnify the company by simply paying them for the repair costs. The 1901 agreement says that Iran will protect the infrastructure, which it failed to do. In any case, those payments were addressed in Article 23 of the new agreement.

It does not appear to me that Section II (D) demonstrates that the contractual stipulations you mentioned (training/hiring of locals and establishment of medical facilities at AIOC sites) were violated. That part of the document also says nothing about infrastructure investment. Maybe I am missing something but I have read it three times now to make sure.


I don't see how Section II (D) could be any clearer. The agreement stipulates a reduction of foreign staff in Article 16 (III), and the document shows a substantial increase. The agreement stipulates the development of sanitation, health services, and housing meeting the most modern standards found throughout Iran in Article 17, and the document shows that workers live in unsanitary tent cities.


> Are we talking about the D'Arcy Concession? I ask because none of what you are describing is found in the text of that agreement.

The agreement was re-negotiated in 1933, according to terms that the grandparent pointed out. At this point I have to wonder if you are being disingenuous on purpose. The rest of your comment is a moral appeal making a case for why D'Arcy "deserved" the profits. You have to pick a lane, are you arguing on the basis of who "deserves" a countries natural resources, or from the point of view of adherence to contracts and agreements?


The financial risk of the initial investor is a reasonable justification for his profits. This applies to D'Arcy as well as AIOC. Their cut of the concession deal was reflective of their risk premium.

Iran didn't take the risk -- or the cost outlay -- of building rigs, importing engineers, etc. The idea that the British were somehow exploitative by resisting renegotiation is ignorant of this fact. Both the 1901 agreement and the 1933 agreement laid out responsibilities for the

Article 16 of the 1933 agreement covered the introduction of more Iranian nationals into the petroleum business, which AIOC did in fact carry out. Article 17 talks about sanitary and public health facilities for workmen, which sounds like bathrooms and medical tents/clinics to handle workmen's injuries and treat their families.

However, I see nothing in the 1901 agreement nor the 1933 agreement that says AIOC will invest in the general infrastructure of Iran. That is what the grandparent said ("generally contribute to the development of infrastructure in the country"). That idea is at best a false pretext for breaking the deal, and at worst just a blind restatement of something on Wikipedia. Actually, the text of the 1933 agreement says that AIOC would require Iran's consent to improve AIOC's infrastructure (including aviation and telephone).

If I'm wrong, find it in the text of the agreement (https://link.springer.com/content/pdf/bbm%3A978-3-658-00093-...) and paste it into a reply for all to see. If I'm right, stop calling me disingenuous.


Fortunately, the specific violations have been compiled by the world bank, as pointed out in a sibling comment.

I just have one part I found incredibly funny

> The idea that the British were somehow exploitative by resisting renegotiation is ignorant of this fact

If overthrowing a democratically elected government is not exploitative, I don't know what is. "Resisting renegotiation" is a very Orwellian way of phrasing such. I'm guessing something along the lines of "we will topple your government and hand over absolute power to a brutal dictator if the terms are violated" was also part of the the agreement?

What about all the other Iranians, the ones that had nothing to do with APOC or oil or the government. Did they also "deserve" their fate?


I would consider the construction of housing, sanitation and public health to be "general infrastructure", but if you prefer a different description, I won't argue. It strikes me as a bit of a moot point anyway, because the obligations to improve its workers' living conditions are clear and those obligations were not met. It doesn't matter whether you want to call them "infrastructure" or not.


Strange analogy you take. One thing is a commercial dispute, another is an act of war.

Commercial disputes should be not solved by waging wars.


Why should a treaty signed by Obama be expected to be upheld by Trump?

I would have sworn there was such a treaty... I think the Europeans still abide by it? Do you agree with Trump's rejection of that treaty?


[flagged]


> I expect you to be cheering when Iran assassinates Trump for reneging

Trump pulled a lever that was explicitly listed and offered in the treaty. It was not "reneging" on the deal any more than using a backout clause is reneging on a home purchase. Yes, it is rarely used, and yes, you may piss off the counter party who was really hoping for their payday, but the fact remains that the possibility is explicitly listed in the contract.


Assassinating Soleimani was definitely not explicitly listed on any treaty.


> I expect you to be cheering when Iran assassinates Trump

I understand that the HN zeitgeist is not circumspect about topics like Iran and petroleum because there is a strong distaste for anything that seems imperialistic here, but that's not a reason to make uninformed arguments and accuse me of cheering the idea of someone's death. I'm making some fairly informative and historically accurate comments. If you disagree with the substance then please do so without accusing me of being inhumane.

> Governments have responsibilities to their own citizens, not foreign corporations.

Governments ensure citizens' prosperity by protecting economy and trade via foreign-facing agreements. The beneficiaries of smart foreign trade are the citizens themselves. Iran is an example of this; its government's budget has been funded almost entirely by petroleum revenue for decades.


> If you disagree with the substance then please do so without accusing me of being inhumane.

I'm not accusing you of being inhumane. I'm wondering if you would apply similar moral standards when the shoe is on the other foot, or if you are a hypocrite.


Wait a moment, you're equating a change of administration (Obama - Trump or any preceding transfer of power in the US) with a change of governance, as in a structural change in the system of governance (which is explicitly spelled out in the comment you were replying to).

It's particularly odd that you make this argument given Trump's unilateral withdrawal from the P5+1 agreement as well as many other bi- and multilateral arrangements. If we're just going to rely on realpolitik, why not accept that the oil companies also made bank out of Iran for a while and hey, nothing lasts forever?


> Wait a moment, you're equating a change of administration ... with a change of governance

In the context of a monarchy, that's the appropriate way to consider it. When Reza Shah Pahlavi took power from the Qajars, the deals with the British remained. There is a level of continuity of governance that exists in the developed West but doesn't exist in the Third World. Ecuador for example was until recently an OPEC nation, but it has had twenty constitutions. However, it is in the nation's best interests, economically speaking, to ensure a peaceful transition of operation across those political perturbations. (I am reminded of software revisions that do not alter the API, so that the software can be treated like a black box and mated to applications without a rewrite of API calls every time revisions are made.)

A current example would be Argentina, where the return of Cristina Kirchner to the country's executive branch has spooked foreign investors (including miners and petroleum companies) because she froze the dollar during her presidency about a decade ago. Ultimately this has a very damaging effect on the country's economy and its people's ability to eat.

In present-day Argentina, we see economic investment chilled by a new administration that operates under the same constitution but endangers foreign investors. In early-20th-century Iran, we saw economic activity boosted by continuity of international agreements across several different monarchical administrations within two separate dynasties.

Please choose the correct lens when you are evaluating a third-world country's economy. What we all learned in social studies class about the differences between regimes, governments, etc.....well it is not necessarily applicable to developing nations where change of administration is frequently carried out by coup or the establishment of a new constitution that is as ephemeral as the previous one.

I tried to communicate this succinctly in my earlier post, quoted here:

> Are you suggesting that all governmental responsibilities ought to be thrown out the window every time power changes hands? This is not a realistic perspective in most of the third world, where power changes hands frequently and change of government is often established via change of regime.


Your extraneous mentions of Argentina are jarring. First, please don't attempt to sell yours as the unanimous view of what's happening to Argentina, who damaged it and who attempts to recover from the damage, and the dollar reserves/policy. Second, we do have a more or less stable constitution (the latest interruption to our democracy, via coup d'etat, was backed by the US in the 70s, by the way). Comparing our democracy to Iran's government is bizarre.


Please choose the correct lens when you are evaluating a third-world country's economy.

You're not the arbiter of correctness, and that was a lot of expostulation to avoid the point I made above.


You are conflating not investing in a country with overthrowing a democratically elected government. Are you really arguing that it was OK for the US and UK to overthrow Iran's government because otherwise foreigners might not have invested in Iran, or otherwise threatened their investments? Truly, this is what they call "capitalism with a human face".


The oil rigs, refineries, and pipelines were the British resource, not the oil.


Then surely all the British were asking for was a fair market price for the nationalized equipment, right?


British companies would not have purchased/built that equipment without the resource access that had been established via contractual agreements with the Iranian government.


Sure. But the Iranian government has sovereignty, just like the US government. That means that they cannot be bound by any contract if they decide to change their mind, the most that can be done is that you might press for a fair market value.

Why do you hold the Iranian government to a standard no other government in the world is held? Would you expect the American government to be forever bound to treaties signed by the British, or to agreements made by the executive alone ranging from two decades to a century ago?


Not strange at all when you consider the Iranians had absolutely no means of extracting it themselves.

They gave willing access to their natural resources in exchange for a relatively massive revenue windfall then seized the sizable British capital investments with no compensation.


Right, so if BP drills for oil in South Texas that means that the Texas oil reserves should be under their control? This viewpoint reeks of racism and Anglo-Saxon exceptionalism.


Those investments were made in a country which was unstable so I don’t see how these companies are owed anything back.


Buying Facebook ads and making fake accounts is not "hacking".


Gaining unauthorized access to the DNC's and RNC's email systems and Podesta's Gmail account is "hacking."


Here's one idea: Make the index open and public, set up a public body to update it, and allow anyone to build a frontend. Access can be charged per query. Other companies/entities/governments will be allowed to mirror the index for essentially the price of data transfer.

This is just something I thought up right now, and sure, there might be problems with the approach. But to pretend like there is absolutely nothing that can be done betrays a willful lack of imagination.


The way that people underestimate the scale and complexity of Google's indexing always makes me chuckle. I'm especially fond of the idea that other parties will just wantonly copy it, like for research purposes or something.

Also I'm sorry to inform you that the way the index shards are produced, arranged in memory/on disk, and the manner in which they are queried and ranked are inextricably linked. You can't make a search-neutral index format that can be queried economically.


Are you arguing that it's somehow impossible or technically unreasonable for a startup search engine to piggyback off google's search index in a similar way to how duckduckgo piggybacks off bing's search index?

And that the very idea that this could be made possible through law makes you chuckle?


I can't answer for OP but I can say it makes me chuckle too.

If the solution was purely to force Google to sell access to their index then yes it seems possible on the surface.

But as mentioned index and ranking are inextricably tied together.

Even if they weren't, no other organization is going to be able to produce search results comparable to Google using their index. You're underestimating what goes on under the hood.

So then the answer (often in these conversations) becomes to open up the ranking algos too.

The problems with that are numerous so I'll just point out some of the bigger ones:

- Arms race: Search is a constant arms race between providers and 3rd parties trying to game the system. The minute you make the algos public, gamers win the race. Search result quality returns to the way it was in the 90's and stays that way until someone else comes up with proprietary algos that work (but is that even legal at this point in our thought experiment?)

- Motivation: If search is open and you therefore can't directly profit from your efforts to improve it (because you automatically give away anything you create to competitors) where is your motivation to keep innovating?

- It's harder than you think: Truly, there's so much more going on in modern search indexing and ranking than you likely realize. The chances that some new organization (especially a gov organization) given access to Google's black box as it exists right now would be able to maintain search result quality for any significant length of time is essentially zero.

But let's imagine that it's as easy as many people think... Wouldn't the solution then be to build a public alternative rather than effectively killing what we have now?


>But as mentioned index and ranking are inextricably tied together.

I'm pretty sure they're quite extricably tied together. I'm almost certain google's engine weights the different ranking variables (e.g. page speed) differently depending upon context. Why not expose those variables to other search engines? Well, it would kill google's search engine dominance - if you're concerned with that...

Unbundling isn't technically infeasible and it would create more competition. This would help with the arms race alluded to. What if another search engine used google's index to build a more spam free index? not good for google but great for Joe public.

>Motivation: If search is open and you therefore can't directly profit from your efforts to improve it (because you automatically give away anything you create to competitors) where is your motivation to keep innovating?

Nothing saying that they can't make money from the users of their index just as Bing makes money from DDG. However, is there any reason their search engine shouldn't compete with other similar offerings? Maybe somebody out there does it better.

Well, other than "we've got an unfair advantage and wed like to keep it please"

>It's harder than you think

Actually it's probably a LOT easier. This idea is a direct attack on google's power and the easiest response they can make is "too difficult. not possible". Not "this would fuck us in the bottom line". Simply "we can't do it, who are YOU to tell us it's possible? "

fwiw if you look through history similar reactions were made to attempts to regulate pretty much all utilities. Then it happened. This kind of response is kind of an expected part of the process. Most recently it happened in the UK when utilities and banks were told to open up API access to their data. Same claim you made.

Part two is when they tell you "it's not fair!". It's coming.


Bing's index and algos are not available to DDG, there's no comparison there. DDG uses Bing's results, they can't see how they're produced. Incidentally, Google offers a similar API.

> Actually it's probably a LOT easier

Can you support that claim?

Just the scale alone is mind boggling when it comes to search.

Then throw in natural language processing, contextual signals, hubs and authorities, content categorization (which grows ever closer to looking like actual understanding), machine learning, a host of other basic and ever evolving quality signals that exist both in and inter-dependently of one another, the more complex signals that arise from the above and on and on.

Search is hard. Even the most casual of Googling (or maybe Binging would be apt in this case) will provide you with endless info about how hard it is.


"Search is hard" deliberately misses the point. This isn't about whether search is hard it's about whether decoupling search engine from search index is hard - whether the APIs used by one could be used by others.

This trick you're echoing is, incidentally, used every single time a government comes looking at unbundling opporunities. I remember distinctly how Microsoft claimed it was "too hard" to decouple office from windows. The banks in the UK made the same claim. They were all equally ridiculous and all equally self serving. There is a lot of precedent here.

Google will pretend that it is "impossibly hard" to expose their internal APIs as well, just as every other company did. It would be surprising if they didn't.

Search is hard, yes. A lot harder than exposing APIs, isn't it?


Bing doesn't make its search index public. It has a query api, where you can provide a search query and get Bing's results on a per-query basis.

The closest comparison to what this person wants is Common Crawl, which is minuscule compared to the big players, and is a 100TB gzipped download that gets updated monthly.


Bing provides API access to its search index. IIRC the type of queries you can do are a bit more sophisticated than just "what's available via the Bing search engine" (e.g. select a market).


That's not index access. You are getting ranked results. Index access would give you the posting list for the term "moose", or the intersection of the posting lists of "moose" and "caribou", or whatever.

You can't make a neutral service that returns ranked results, because that's contradictory.


> The way that people underestimate the scale and complexity of Google's indexing always makes me chuckle.

The same approach that works for Google can also work for a public body supported by public funds. It can also use much of the same systems and maybe even personnel that Google currently employs for this purpose. I'm essentially asking for the search division of Google to be appropriated by the government.

> Also I'm sorry to inform you that the way the index shards are produced, arranged in memory/on disk, and the manner in which they are queried and ranked are inextricably linked. You can't make a search-neutral index format that can be queried economically.

I'm not asking for a search neutral index - public Google just needs to be (largely) open, transparent and accountable about its ranking algorithm. That's why the stipulation that other entities be allowed to mirror the index - they can optimize the index for their own purposes and rankings on their own hardware.

> I'm especially fond of the idea that other parties will just wantonly copy it, like for research purposes or something.

Not for research purposes - other entities will be free to copy it for purposes like sovereignty (governments), less/more censorship or to build their own systems on top of it for profit, or any other purpose.


There are currently zero governments and only about 4 commercial entities possessing a datacenter large enough to do the job.


https://en.wikipedia.org/wiki/Utah_Data_Center

> designed to store data estimated to be on the order of exabytes or larger

While google says:

> The Google Search index contains hundreds of billions of webpages and is well over 100,000,000 gigabytes in size

https://www.google.com/search/howsearchworks/crawling-indexi...

Seems like the government already has the expertise and equipment required to handle data at Google scale :)


Having disk space is only a small part of being able to make use of something like a search index. Arguably the least difficult part.

One of your suggestions was

> That's why the stipulation that other entities be allowed to mirror the index - they can optimize the index for their own purposes and rankings on their own hardware.

And the point is that there's nobody who can do this outside of Google, Microsoft (who also does), Facebook, and Amazon.

Not to mention the problems of actually getting the data. You're at the scale of data where trucks of disks are faster data transfer than cables unless you have direct fiber backbone connections.


99% agreement, except for the amerocentric viewpoint. I think it is likely that Baidu has the scale.


I knew I was missing someone. Yes, Baidu (who also already runs a large search index) could probably do the same thing.


A giant building full of hard drives is NOT a Google datacenter.

You're just comparing two storage numbers without taking anything else that running Google at a global scale requires.


The NSA also monitors most of the worlds communications in near real time. A building full of hard drives is also completely useless without some sort of reasonably decent search and indexing capability, so I'm pretty sure the NSA built something for the task.


Doesn't the NSA have a bunch of massive datacenters?

Regardless of that, the datacenter can be appropriated by the government too.


No, the NSA has one facility that would be a small/medium datacenter in the big league, but only if you assume that the NSA is as efficient as Google, which is a bit of a stretch.

NSA Utah: 65 MW Google Pryor, Oklahoma: 340 MW


Megawatts are indicative of compute load, not storage load. I can definitely believe that Google is doing more compute than NSA, but that sounds more like a difference of need, not of ability.


What do you think the query pipeline looks like?

I can assure you that it's not mapping each query down to a single-sector disk read off an inverted index.


?

I think the query pipeline for NSA (relative to the scales of Google's query pipeline) looks like absense-of-query-pipeline. Hence NSA using less compute and thus (the reasoning goes) less power consumption.


Presumably that Google data center does a lot of compute intensive, non search related stuff - like GCP for one.


Another thing that people persistently misunderstand is the scale relationship between GCP and the rest of Google.


Even if all you say is true and it is truly impossible for the government to replicate any of what Google does, the point is moot. If the government is going to appropriate Google's index, might as well appropriate the datacenters too. Really, whats Google going to do with them once search is gone? According to you, it is the only thing they have have running there.


Might as well appropriate the engineers, too, and chain them to their desks and force them to keep everything working.


Truly an absurd comment. The US government is the largest employer in the country, with something in the order of 10 million employees. Are they all slaves chained to their desks?


It also does a lot of storage intensive, non search related stuff, like Google Cloud Storage.

Every Google data center does...everything.


Any tech corps shouldn't be owned by US Gov. Please.


Governments can do that right now too. EU is trying to do that right now with Qwant.com where European Investment Bank invested €25 million.

So, I'm curious will you start using Qwant.com from now on?

https://www.qwant.com/


How are my personal decisions relevant? I'm calling for a government appropriation of the search division of Google.


If you say you want it government run after stealing it (lets not mince words here) and aren't using a government run one right now kind of undermines the sincerity of your arguments.


> aren't using a government run one right now kind of undermines the sincerity of your arguments.

First I never said I'm not. Either way it is pretty irrelevant to the discussion.

If, in fact, I am not using a government run one right now, maybe it is precisely because the government hasn't yet stolen Google's? If the government already had something as good as Google, clearly there would be no point in them stealing Google's, would there?


There is certainly a huge difference between setting up extractive institutions designed for the benefit of people living half a world away, compared to extractive institutions set up for the benefit of people living a few hundred kilometers away. In the latter case, you have a much greater chance of seeing some of that wealth again, and of being able to influence the decisions of your rulers.


Do you have any evidence of this? I doubt the poles would agree with you.


Do you have evidence to the contrary?

Logic dictates that any wealth or resource that staysin the community is more likely to benefit that community as opposed to if it is shipped to a different continent and never seen again.


Yes, the Germans invaded Poland, Russia, and most of Europe and extracted lots of resources at a significant disadvantage to the Poles. Japan invaded Korea, China, and much of Southeast Asia and killed hundreds of thousands of people for the sake of colonialism. I don't think the locals were happy about that. The United States relationship to the Native Americans. Mongol conquest of central Asia. The Zulu conquest of their empire. The Italian and French occupation of North Africa. The Ottoman relationship with the Armenians. The relationship between South Sudan and Sudan. Iraq, Iran, Turkey, Syria and the Kurds. Israel and the Palestines. All of Europe and the Jews. I can name historical examples of neighbors exploiting neighbors all day.

You can't just say "logic dictates" when logic dictates no such thing. I'm not saying your wrong, or there is not correlation, but there is no clear pattern between how far off an occupier is and the treatment of the occupied from what I can see.


I'm not sure that this is a "huge" difference, especially when compared to the clear difference between extractive and inclusive institutions.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: