Having been in this position before, I can tell you that maintaining large patch sets increases friction against submitting anything upstream. Even if this was a Google-only CPU (it's not), that'd still be an argument for merging upstream.