Bling [1] was quite similar to this. The idea is that rather than use and build values, you can use and build expressions instead. With those expressions, you can generate whatever code you really need (including GPU code), but you can also walk them to do things like symbolic differentiation (in the style of Conal Elliott) or compute an inverse.
A lot can be done with C#'s support for operator overloading and extensions methods to make this look prettier.
I use System.Linq.Expressions all the time, first time I used them in Bling and even had a nice discussion about it with Jim Hugunin and company when he was still at Microsoft. I'm actually using it right now in my current work on live programming!
Still, you have to put a lot of boilerplate around it to make it viable. Bling wraps everything in expression trees and as a backend step will go to Linq expression trees for anything that needs to run on the CPU. However, it can also go to HLSL to generate GPU code, which is quite useful!
I'm in desperate need of something like this from C# to SQL that isn't LINQ to SQL or Entity Framework (they're very centered on MS SQL Server and I don't like their ancillary workflow).
I think LLBLGen still has the most mature Linq > SQL translator and supports most major databases, as they have an intermediate query model between the LINQ expression trees and database providers. Unfortunately only available with generated code, no code-first support.
What's the endgame here? Building a competitor to Mathematica/Matlab based on MS technology? Why C# and not a scripting language? Is it supposed to be combined with a numerical toolbox, so that's why a fast compiled language is used? I could see the value in that, the mismatch between Matlab and HPC is annoying, i.e. porting numerical code from Matlab to Fortran to make it fast, is not fun. Then again I'm not sure whether the C# runtime is efficient enough for HPC purposes, there isn't really much research going that way AFAIK - could be an interesting topic since I'm not convinced that the current trifecta of C(++) and Fortran is the final answer.
I looked for that once but couldn't find anything. However, it's quite easy to write and manipulate expressions in Haskell. The type system and recursion are a good fit for that kind of task (also, there's an expression parser).
If you only need it for some simple things, I'd recommend writing it yourself. It shouldn't take too long.
A lot can be done with C#'s support for operator overloading and extensions methods to make this look prettier.
http://bling.codeplex.com/