Bravo to Google for this show of corporate leadership. Their pledge to purchase renewable energy to match their real-time demand really is impressive, especially the note that it must create new renewable generation (discussed on the Google blog announcement not this article) .
It is also prudent, as these long-term power purchase agreements (PPAs) with new renewable generators will be very competitive, probably much cheaper than wholesale power prices, and be fixed into the future, avoiding price fluctuations from e.g. future gas supply shocks.
I wonder, would it be feasible for them to use wind power to run time insensitive batch jobs in the daytime - crank down the processing power on these jobs when there's less green energy around?
Google has insane optimization for power consumption at data centers, if there is something you can think of, they will have either done it or will be trying it.
For many a datacenter - balancing the loads across the three-phase power supply is one area that makes one of, if not the biggest difference in the bill/power charged for using. Though that for many a tech, is an area they are more inclined to overlook.
Then quality of the power (clean sine or noise spikes) auditing, then the factor that UPS's do square wave. Though that's more stability and indeed longevity of your kit plugged that appreciates those nuances.
It is one big rabbit hole and can imagine the likes of Google having dedicated teams to focus just on power optimisations as the dividends at that scale - more than pay for the team.
When you deploy a datacenter at this scale you contract for an amount of power and you're basically obligated to use it. The trick is to get more compute out of the same amount of power.
By the way if you still have AC UPS systems that is probably your #1 problem and you definitely will not benefit from some fancy machine learning thing. Just get rid of most of your AC systems. For example you may benefit by adopting Open Rack 48V-to-point-of-load scheme that uses an in-rack DC UPS.
Total, more the company inhouse data center type affairs - big, but not your cloud or google size affairs. Then doing one AC to DC conversion instead of all that overhead per system along with the associated heat and centralising - pays for itself.
The cut-off for that level of work I'd say is if you design your own servers over just speccing from a vendor level is when you would be doing this. Upto that, it's still vendor off shelf. Though been a while and an option some vendors may now offer at lower scales these days.
Do they openly share these advances & techniques? I know, it's a competitive advantage so I wouldn't fault them too much if they don't, I'm just curious.
I wonder if preemptible instances could be priced or available this way. They could choose to run idle or contract in case of expensive or dirty power.
(1) Existing power usage doesn't work that way. You're basically on the hook for a predicable amount of usage. Of course, this could change, but the inertia behind this paradigm is high.
(2) Google has a bunch of hardware with relatively fixed costs that they'd prefer to run 24 hrs/day, 365 days/year.
A big challenge for Google, and any large DC operators in general, is utilization. Ideally you want the entire fleet working at 100% all the time, so there's no real benefit to ramping up or down with energy supply.
> avoiding price fluctuations from e.g. future gas supply shocks.
Yes, especially in Europe where the dominant natural gas supplier, Russia, has shown itself more than willing to use their position as a political hammer. And I know, that's not just a Russia issue, other countries do the same, they're just the most applicable when talking about gas in Europe.
The Hamina data center is close to Russia without being in Russia. I guess it's the closest Google is comfortable at being when serving Russian customers.
In the north data centers can double as heating power plants when coupled with district heating. It's unfortunate that Google puts the excess heat into the sea. Yandex has a big data center in Mäntsälä. Their cooling water is used for heating the area. There are also data centers in Helsinki where heat is used in district heating.
Hah that's amazing. I remember when I did bitcoin mining (only 1 GPU) and thinking it was getting noticeably warmer upstairs in our house, and how actually the bigger miners could use the heat to heat buildings.
Great to hear this is being put into action. Computing -> heat.
This is not true. A computer heater can only ever be 100% efficient, but most electric heaters are actually heat pumps and can be up to 400% efficient, depending on conditions. Heating your home with bitcoin miners (or any other computing device) is a staggering waste of energy.
A heat pump is basically an air conditioner running backwards - it runs uncompressed refrigerant through a radiator outside to pick up heat from the outdoors, and compressed refrigerant through a radiator indoors to release the heat. Because they can move more heat than they dissipate (and some of the dissipated heat is indoors anyway) they can be more than 100% efficient. (edited to fix compressed/uncompressed sides)
But I'd venture to say the statement "most electric heaters are heatpumps" is in fact false. There are a lot of heat pumps in the world, but there are also a lot of pure resistive electric heaters. I suspect that most things people call electric heaters are resistive electric heaters.
If a Stirling engine can be operate at 50% efficiency and a heap pump can operate at 400% efficiency, could we take a source of heat (e.g. heat side product of computation instead of burning fuel) use it to power the Stirling engine and use the movement of the Stirling engine to power a heat pump and thus turn computation into house heating at effective 200% efficiency? (under whatever assumptions were made in the claim of 400% efficient heat pump; I assume the temperature differential is key; I also assume that the optimal cold side for the stiling engine and for the heat pump is quite different, but you can tap one in cold air and the other in deep ground)
A heat pumpt doesn't generate heat, it /pumps/ it from one location to another. It can only achieve above-100% efficiency if you don't include the source of the heat in your calculation.
For a more practical example, consider a (simplified) geothermal heating system. It consists of a probe that's drilled some 10-15m into the earth, a radiator in your living space as well as a pump and piping connecting the two. The earth's temperature surrounding the probe is relatively constant at 10-15 degrees C.
In winter, when the outside temperature falls below those 10-15 C, you pump warm water from the probe into the radiator. Using the example numbers, 100W of electrical energy might provide you with 400W of heating output. The 100W has no part in generating the heat though, it only moves it from the warmer probe to the cooler radiator. The reverse applies in Summer, when the surface temperature is higher than 10-15 C.
What you're describing exists in the form of district heating. Heat is generated in a central location (e.g. as a side product from garbage incinerators), and a heat pump is used to transfer the thermal energy from that location into a bunch of surrounding houses. But, in any case: the whole process only makes sense as a way to capture excess energy from the heat-generating process; and you are always limited to (at the theoretical maximum) capture all of the excess energy output, but not one Joule more.
No, what I'm describing is using a heat source to produce mechanical energy (step 1) and then use that mechanical energy to power a heat pump that works as you described (step 2).
Heatpump efficiency is quoted versus heating with resistive electric heaters. In northern europe most modern houses are heated by heatpumps and good ones achieve 5-6x efficiency as opposed to "dumb" electric heating. It's basically a reverse fridge - you cool down outside and dump the collected heat inside. When outside temps drop the efficiency drops dramatically though. Google "heat pump COP curve"
Confused, could you explain? A heater that puts its energy into something other than heating is just being inefficient at heating, right? Shouldn't you want it to put 100% of its energy into heating?
To very good approximation, 100% of the energy used by a contemporary computer is turned to heat. The amount of negentropy generated by the computations is utterly dwarfed by the increase in entropy in the computer's surroundings. Computers are space heaters that happen to do some calculations as a side effect.
I do realize that and I still don't get how doing computation is supposed to increase the energy efficiency of heating. Are you trying to reduce wasted energy while heating by diverting it to computation, or are you trying to avoid wasted energy in computing by utilizing the otherwise-wasted heat? It seems like you're optimizing the cost efficiency of computation while claiming it improves the energy efficiency of heating...
Computing is heating, but with benefits, to a rough approximation.
The components inside the computer do something with the electricity, which results in it being turned into heat. So by doing useful computing and using the created heat as a result for heating something, you've effectively made the heating free, to the degree you can use the computing for something useful.
OK, I think I see why we're seeing this differently. The only way this actually saves energy (instead of merely money) is if, later in the future, you would've actually wasted the heat that would've be produced if your initial goal had been computation. In that case, yes, it's true that doing the computation now and using it for heat is better. My assumption was that you would be already using any heat you generate from computation for heating regardless, so it didn't make sense to me that, in other situations where your goal is only heating, adding computation to the problem would save any energy.
(Sorry this is complicated to explain, I hope it makes sense. Regardless, I get your point.)
Well, not really, let's say we want to heat a room to 25 degrees Celsius. If we are converting 100% of the electricity used to heat, it will take X energy. If we use the energy for computing instead, and use the waste heat to heat the room, it will take maybe a little more energy. But the thing is, the computing is valuable, and with the money we can gain by "renting" the computing, we can buy more electricity, so we come out on top.
Now, like someone else commented, if you really want to heat an environment, the traditional resistance heating is not the best solution. With a heat pump, you can cheat, and get an efficiency of over 100% (because instead of using the energy to heat the room, you are using the energy to take heat from somewhere else and also put it in the room).
That's amazing, but what sort of computation does it do while heating? Are you supposed to queue up your rendering until it's cold outside, or does it mine bitcoins?
It's not the money. Dedicate the computational power to @Folding and it would be worthwhile even if they also paid people. Personally if I had to choose between generating data for medical research or hash functions for proof of work, I'd go for the medical research. Though I'm not sure we really have to choose-- aren't there coins in the works that are trying to address the massive power consumption issue?
No, because you need to be able to exploit a temperature difference to convert heat into other forms of energy.
Conversion efficiency is proportional to the difference in temperature, so for most wate-heat scenarios, electricity generation is either non-functional, or barely functional, but impractical. In a lot of scenarios you end up interfering with the efficiency or longevity of whatever you were trying to cool in the first place.
>Hah that's amazing. I remember when I did bitcoin mining (only 1 GPU) and thinking it was getting noticeably warmer upstairs in our house, and how actually the bigger miners could use the heat to heat buildings.
I switched to a Chromebox as my daily driver 2 years and change ago because my bedroom would be about 1F higher than the other rooms at idle and I saw it 11F hotter one time after several hours of one of the BattleField titles running dual GPUs.
The added benefit is, even with the PC running at idle 24/7 the Chromebox would have paid for itself right around the 2 year mark in energy savings at a full load.
I remember being able to mine on a single Nvidia GPU on my laptop, significantly less efficient than an AMD one (at least at the time) and giving it up as pointless when it only generated a single bitcoin in a week. And then a year or so later being thrilled when I read an article that the price skyrocketed to $200 and I remembered that one coin, and sold it. Oh well!
I looked up a few days ago that I had about 23 mined BTC, and sold them for about $1-2. Not going to lie, it stings a little (each one is now worth more than a year's income).
This is one of the reasons why I only run BOINC on CPUs during summer and on GPUs and CPUs the rest of the year. My 10 year old i7 have been running 100% mostly 24/7 for 10 years with no problem, while 2-3 graphics cards have given because of the heat
It is amusing how in their goal to be carbon neutral they're just heating the sea directly, cutting out the atmosphere as a middle-man. I'm not sure what the numbers on it are though. I guess heating the sea will only be once but having CO in the atmosphere will provide recurring heating.
I imagine it's the difference between using the oven in your home and installing insulation. The oven does directly heat your home a bit, and the insulation does not, but the insulation will have a much bigger impact.
Anyone find anything that Google does these days to be aggressively competitive? As a one-man startup guy, I feel everything I do pales in comparison to what Google are doing and I feel nothing more than /eclipsed/ by them. I have found myself wondering: "If you can't beat them join them" on many occasions. I probably should just merge into Google one day and stop my worrying?
I'm a startup founder, and I've never understood this worry.
I know of scary few products by Google (or one of the other tech bigcos) that successfully wiped out an up-and-coming startup.
They did acquire quite a bunch, but can you name a successful Google product that killed a startup without acquiring it? I'm sure there are some but it can't be much. Docs, Android, Firebase, Maps all came out of acquisitions.
I mean, sure, they're super competitive, but my impression is that that competitiveness is mostly targeted at Amazon/AWS, Microsoft, Facebook, etc. I mean GCP is mostly just Google reimplementing the most popular AWS products with less cruft. That's super competitive but I'm curious how it threatens your one man business.
Without Google's resources Android, Maps, YouTube, DoubleClick and the like wouldn't be the powerhouses they are today. Take that how you will, good or bad.
Every company has things they structurally cannot do well. Google scales? They can’t do customer service. Facebook has all the network effects? They can’t iterate rapidly. Tesla has the best electric cars? They can’t sell nostalgia.
That’s why people tell you to “be yourself”. It’s a rough proxy for “don’t be other people” which is a rough proxy for “don’t be something someone else is already succeeding at.”
But also, if you are really enamored of Google, maybe that means you should work towards getting hired there!
It's easy: find a niche which is too small for Google or any other large company to bother with. There are plenty of those, especially if you look for something tied to the physical world (serving real businesses around you) and if you avoid the SV echo chamber (which means HN much of the time, too). In other words, don't try to be another Google, Facebook, Uber, or Bird. Do something specialized.
Google can only do so many things. Every avenue needs to have a significant impact. Google is also a SV echo chamber. They have a lot of blind spots. Lots of room for smaller startups to carve a niche. After get large enough to get Google's full competitive attention another story.
People once felt the same way about Microsoft. There’s ways to be competitive with Google for whatever you’re interested in. You can compete by having good customer service for your product. They’ve given that up to the algos.
I know I personally highly value the ability to talk to an actual human if I’m having a problem.
As any sector matures it becomes harder and harder to truly "disrupt" (not talking about "fake disrupt" by being better because you take shortcuts even if it breaks the law). You see it in many other sectors, consolidation is key. Standing out, while not impossible, is improbable. This is not meant to discourage people from trying, you never know until you do.
But if big IT has learned their lesson they won't wait until you're too big to run you over. It's why everyone is on acquisition sprees these days, or simply incorporate tech and ideas into their products and use their sheer market share to drown smaller competitors they borrowed the idea from.
A niche may be different since no company can cover everything. But it's still uncertain when they choose to go there and go head to head with you. Google didn't overtake MS by playing in a niche. So I guess it depends on what your targets are.
The cost to move atoms and bits everywhere in the world is near zero.
This means that you could be the best in your field for hundreds of miles around, but your neighbors might still go on Amazon and order a product manufactured in China.
You are correct, google is trying to cover all sectors they can, and offer tools to join them even at a loss. After they own everything, they can start raising prices, dictate web standards and so on
I think we can make useful distinctions here between Google services.
- Google Search/Ads and Google Cloud are money makers. They are constantly improved, handled with care and have prices comparable to the competition, maybe slightly lower
- Chrome is costing massive amounts of money. Between the massive development effort and the big worldwide marketing campaigns Google must have spent billions on Chrome in a play to protect their other services and dictate web standards. Android also belongs in this category, dictating a massive portion of the mobile market
- GMail/GSuite, Youtube, Google Home etc probably about break even. Like the previous category they are strategic plays aimed at controlling a market, and they are great for the brand. But since their markets are less valuable they are the step children
- lastly we have the experiments. Google has cut most of them, and you are probably better off not using them at all
A monopoly that keeps prices artificially low for years, drives everyone out of business, and then raises prices to make up for the years of lost profit -- I'd like to hear one example - it's never happened as far as i know.
Most monopolies have been government granted - railroads through free land, AT&T by creation, etc.
You could argue about Microsoft. They started out by offering MSDos as a low cost option and now own close to the entire OS market.
Google is undercutting everyone in the mobile OS market. What once was a varied market died completely because nobody could compete with Android's "it's free if you agree to preload a bunch of Google apps" (iOS isn't really on the market since its Apple exclusive, LG can only reasonably choose Android). Google makes all that lost income back because this creates a monopoly for the Play Store, and massively helps other Google services.
Not really textbook examples, but that's because the textbook example of selling at a loss to kill all competition and then raise prices is illegal in most cases
Microsoft certainly drove almost everyone out of business, but I don't think you can accuse them of raising prices to cover for lost profits - a OEM Windows license is pretty cheap compared to other paid software, particularly before they started adding ads to it.
>This Note argues that the current framework in antitrust—specifically its pegging competition to “consumer welfare,” defined as short-term price effects—is unequipped to capture the architecture of market power in the modern economy. We cannot cognize the potential harms to competition posed by Amazon’s dominance if we measure competition primarily through price and output. Specifically, current doctrine underappreciates the risk of predatory pricing and how integration across distinct business lines may prove anticompetitive. These concerns are heightened in the context of online platforms for two reasons. First, the economics of platform markets create incentives for a company to pursue growth over profits, a strategy that investors have rewarded. Under these conditions, predatory pricing becomes highly rational—even as existing doctrine treats it as irrational and therefore implausible. Second, because online platforms serve as critical intermediaries, integrating across business lines positions these platforms to control the essential infrastructure on which their rivals depend. This dual role also enables a platform to exploit information collected on companies using its services to undermine them as competitors.
This is also the same approach that Uber is using - charge artificially low prices for rides to undercut other cab companies, and eventually deploy self-driving cars and increase their prices.
They are claiming it could happen, not that it is happening.
Which is what I'm saying - people claim it happens, but it never has. It's near impossible to corner a market without the government granting the monopoly.
I believe the Rockefeller's General Oil was dissolved as an illegal monopoly.
It used it's massive, over 90% share of the market (and it's vertical supply chain) to deter competition, and made deals with other suppliers to make the inertia of competing a herculean task
While a company that controls the majority of a market may be the text book definition of a monopoly, it's the the legal definition in the US. The US law recognizes that a company may naturally (and legally) gain a majority market share (even 100%) and be left alone. It's only when a company with such control over a market uses that to leverage control over another market that it becomes illegal under US law.
Standard Oil owned 90%+ of the oil refineries in the US (they never did drill for oil) yet that wasn't why Standard Oil was broken up. They were broken up because they were using their influence with oil to control the train industry, specifically, train transport of oil. [1]
Microsoft wasn't threatened with breakup over their operating system monopoly on desktop computers, but because they were using that influence with at attempt to control the web browser industry.
[1] And ironically, it was the breakup of Standard Oil that made John. D. Rockefeller the richest man in the world. He was also responsible for saving the world's population of whales. [2]
[2] Half-serious here. Whale oil was big business, until Rockefeller made petroleum products cheap enough to supplant whale oil as a product.
During the rise of standard oil the price of oil went down, it did not go up. Competitors fought and complained that they couldn't compete because standard was vertically integrated and got it broken up.
Google isn't doing anything new + interesting at all.
I consider Google to be one of the giants that are entirely irrelevant in competitive terms. They almost comically stay in their lane: search, YouTube, search, Gmail, search, search, ads, search, Android, search, search, Google Cloud, search. Every ten years or so maybe they'll add something.
I consider Google to be a company that never has to be considered when I'm thinking about building something.
I have no plans to launch my own massive operating system (what small team is going to do that anyway). I have no interest in doing a common search engine. Competing in the streaming space with YouTube, Netflix, & Co. is a nightmare of licensing & rights problems (and it's very saturated). The last thing I'd ever want to do is run an email service (thankful for those that do it well). AWS, Azure, and 37 other cloud service providers already exist, no need to reinvent that wheel and compete with Google Cloud (they're not the most competitive there anyway).
Google search is 21 years old. Gmail is 15 years old. YouTube is 14-15 years old. Android is 11 years old. Google is gray.
So what else is there? That just leaves 99% of everything else that I never, ever have to worry about Google competing seriously in.
Of course every mature product has is more than 10 years old: it takes 10 years to get to a point when the product itself is mature.
Google is the first company that started to spend money on productionize self driving technology. It takes time to get to market, because the problem is super hard.
If you want something newer that probably won't go away any time soon, Tensorflow is a few years old, and TPUs are also just a few years old. Google has many of the best and biggest number of AI researchers, and it's not shy to use the AI infrastructure in production.
Anyone got a source that doesn't require an afternoon's worth of work to set your privacy options? I clicked through a bunch of menus to try to opt out of personalization and got nowhere and gave up
> This [...] is in addition to the $7 billion the company has invested since 2007 in the EU, [...] today’s announcement was focused on Google’s commitment to building data centers running on clean energy.
Newsletters can be incredibly profitable for media organizations because it 1) gets people to return to your content on a regular basis 2) can allow for more targeted advertising
I personally hate this trend, but almost everybody's doing it these days
These data centers will collect large amounts of personal data. I believe that the future of personal privacy protection issues needs to receive greater attention.
I wonder what has bigger impact on climate, driving a car daily or feeding Google and Adtech industry with all your data, that has to be stored and constantly processed.
Will these data centers still be subject to mass data collection by the us government (under arrangements similar to the prism program described in the Snowden relevations)? Does it matter at all that these centers are placed outside the us?
Just like basically this biggest nit ever, but Snowden revelations didn't reveal anything and nothing was described in them. Snowden exfiltrated a bunch of powerpoints that neither he nor you nor I can understand out of context. The most supportable conclusion you can really glean from Snowden's slide decks is the US intelligence agencies were able to intercept traffic of undersea cables.
They might be (a lot) less bad than other data centers, but unless these google data centers have a neutral carbon footprint, they are not clean, and unless they have a negative footprint they are not "environmentally friendly".
Google's datacenters are already carbon neutral. This is about going one step further and using locally-generated renewable energy to power the datacenters.
No they are not. They use energy, a lot of it. They want to run their data centers off of renewable, "carbon-free" energy sources and made some great strides towards that. But they aren't there yet.
On top of that, renewables aren't carbon neutral per se, either. And the hardware you put into those things isn't free either.
Now you might argue the services google is able to provide with them warrant the environmental costs, but that's another question entirely. And you might argue that google did a lot of things to reduce their footprint (and they did) but it's hardly close to zero, let alone negative.
As long as there are always more humans on this planet, we will have to serve them. Factories, food, electricity can’t be decreased if every decrease per person is followed by an increase in headcounts, which is what’s happening of all western countries since we invented ecology.
In other words, do you have a plan for decreasing actual footprint without having it followed by: “Hey guys, we can afford increasing our population!”
>As long as there are always more humans on this planet, we will have to serve them.
Sure, tho youtube cat videos are not a basic need in my humble opinion, and neither is tracking my every digital move so you can sell me to advertisers.
>Factories, food, electricity can’t be decreased if every decrease per person is followed by an increase in headcounts
Well, food demand will increase, but can we and the comparatively fragile system that is our planet afford that people eat a big chunk of cow every day? How about a big chunk of cow every few days instead?
We can actually decrease factories or rather their ecological footprint, and we can use less electricity (a new fridge uses less electricity than an old fridge, and there is technological advancement to be had, and a lot of low hanging fruits still) and we can can produce energy with less pollution. And we can plant some fucking trees. And then even cut down those trees and bury them in the ground, putting back some of the carbon we took out before. And regulate polluters into oblivion.
A lot of changes would be "hard", like taking away people's daily burgers, and punishing corporate polluters. So they probably will not happen because politicians are chicken shits, scared they will be voted out of office if the price of meat goes up, or some coal mine closes and loses some jobs.
But we can at least regulatory incentivize good behavior and better products.
But yeah, my plan would also include tackling the population issue as a major part of it. Not in a "one child" Chinese way, of course, but I'd strongly deincentivize having many children, starting with no more tax breaks/credits after birth two and creating harsh prison sentences for people who are not willing to pay their child support (which is different to not being able to pay, of course), but also investing heavily in making access to free contraceptives and education about contraceptives available to everybody. But sadly there are far too many evil morons running around preaching abstinence-only, because their imaginary friend in the Heavens doesn't like the feel of latex on the skin or something.
And a lot more out there, I'd also invade the Vatican and try the pope and his flunkies for crimes against humanity. If we can put Serbs and African "leaders" on trial, we should be able to do the same to the pope.
What'd be better for the environment is to not build anything at all, and force people to use existing resources more efficiently through higher pricing rather than buying more resources to run at 1/30th the speed a real programming language would be able to do. Not a realistic proposition, but let's not get taken by the narrative that a multi-megawatt datacenter can somehow be as green as they claim, once you consider the downstream effects.
At Google Next in April, I attended a talk by the C-level in charge of Google's Datacenters, and they bragged about using machine learning to manage/anticipate their power draw, and the subsequent efficiencies those brought. They also bragged about how much they're trying to make these centers powered by renewables.
When an audience member asked if they would be releasing their models that help them manage power, the speaker quickly changed the topic, which I felt told me all I needed to know about the sincerity of their motivations.
The trained models are likely not directly applicable to any other company's data centers. Instead, Google publishes machine learning research. Open access so that everyone can benefit. Here's a paper on Deepmind's reinforcement learning system for datacenter cooling: https://deepmind.com/research/publications/safe-exploration-...
They could have saved Earth and you would find a fault. This is a great initiative, but you're criticizing about not "open sourcing" it. Google open sources plenty of things. The solution maybe heavily coupled to their infrastructure.
And who knows it was probably out of the scope of the presentation and he simply didn't know if they had any plans to publish research or directly open source it.
I doubt Google's being super protective of data center power optimization stuff. Demand optimization sounds like a relatively straightforward software problem for Amazon, Microsoft, etc to solve. Most of the utility would be for the smaller players.
So here's something you can personally do to make a dent to this problem. Most of the Hadoop jobs that you write will involve some statistical summary over a dataset. Find the total, or the mean, median, 90th quantile, whatever. Writing a Hadoop map-reduce job is the single worst way to do this.Almost always, you can sample say a 1000 points, get a kernel density estimate via Parzen & then use a table. All quantiles, order statistics, functions of order stats...these can be hand computed for several univariate distributions & their location-scale families, and your real-life data can easily be bounded above below by these estimates. So you can get to > 95% accuracy just by hand-calculating. I can go into details if you like, but I suspect most of you already know how to do this.
You're exactly right. Hadoop and it's ecosystem are... a very poor replica of the system whose predecessor the respective Google papers were written about. There's glaring efficiency things like not supporting complex encodings (last I used HDFS, at least, it could only do full replication) or the query engines on top not implementing sampling in their aggregates as a default/out of the box, which the Google tools do.
Another way to get huge efficiency gains over using Hadoop is to intelligently memoize results and compute as-you-go. But that requires making some bespoke architecture. However I contend that at any scale where it actually makes sense to use Hadoop (especially in the cloud era), it will be cheaper in the medium-term of 1 year or so to dedicate developer time to make something more efficient than a big daily pipeline of Hadoop jobs
Google’s datacenter power management relies on software features that nobody outside Google is prepared to accept. Most operators expect to have thermal headroom for short CPU clock rate bursts and they don’t want some central daemon downclocking their machines. From what I’ve seen outside Google, most organizations would get much more benefit from focusing on utilization than trying to replicate Google fan speed control techniques. If you still have UPSs and dual AC-DC power supplies in every box and your PUE is 2.0 you don’t need the thing that allowed Google to go from 1.11 to 1.10.
The models in isolation aren't useful. The knowledge to create models like them is where the value lies, but it's only possible to generate value if you can do this in a way that is broadly applicable... which is where building systems management partnerships come into play.
Yes, you can argue the point you made, but it's not particularly insightful because the specific IP employed to govern power mgmt at Google DCs is only truly applicable to Google DCs.
Release of source-code and licensing of said code is typically a very significant consideration for a company like Google.
Even a "C-level" person in the singular sense most likely does not have the authority to approve such a thing. And if you don't have approval, you probably shouldn't talk or speculate about it in a public forum.
Don't hate the player. Hate the legal-liability game.
You think you can drop a model into a new data center with a different architecture, different censors, different workloads, and it will magically save energy?
even if they didn't, seeing the limits of those models in different environments would certainly help the community overall, and if it saves a tenth of the energy it does for Google somewhere else it would be an enormous immediate reduction in energy consumption on a global scale, so it's hard to see what the downside would be.
If one was truly concerned about 'democratizing AI', as companies like Google so often claim, then sharing access to the trained models would arguably be far more effective than just sharing research papers which many companies don't even know how to implement. In fact the large majority of companies that are responsible for energy consumption don't have any data on the scale that Google has, so I would go as far as call this a bad faith deflection from the beginning.
It's exactly because the majority of companies don't know how to implement those models that it wouldn't do any good. This is the "give me teh codez" of AI. Google's models are specific to their environment, and they would be more likely to generate waste (at least in time and money required to implement them) than to reduce it.
It is also prudent, as these long-term power purchase agreements (PPAs) with new renewable generators will be very competitive, probably much cheaper than wholesale power prices, and be fixed into the future, avoiding price fluctuations from e.g. future gas supply shocks.