That's not really an argument, is it? I do understand the maths behind these models, or at least a good chunk of it. I can talk about differentiable functions, softmax etc.

Regardless, going from "it involves maths" to "therefore there are no actual ethical issues" is a non-sequitur. Your conclusion doesn't follow from your premise. I actually agree with you - I don't think current transformer models like DALL-E are conscious or worthy of rights - but what you've presented here isn't something that could possibly convince anyone in either direction. You'd have to lay out why it's not and how many years of study of maths are involved really doesn't seem like a convincing basis on which to decide.

> nobody seems particularly willing to do that

This. The amount of mathematical illiteracy is staggering in ML.

The professionals I've met actually working in ML R&D have basically all been very technically competent, including in mathematics. It's more the people who talk a lot about AI in grandiose and anthropomorphized terms that I was referring to.

> The professionals I've met actually working in ML R&D have basically all been very technically competent, including in mathematics.

Competent at math can mean many different things. Have they taken higher level courses in statistics, probability, optimization and control theory? If not I'd say that they aren't technically competent at math that is relevant to their field, and in my experience most don't know those things.

I work at one of the best unis in the world in a big ML research group and I have not. Unfortunately.

I even know researchers with 10k+ citations in ML that even talk about "continuous maths" and "discrete maths". This pretty much sums up their level of mathematical sophistication and ability.

What do you mean? That's an incredibly important distinction in understanding mathematics for ML in the neural net age. Perhaps a bit of a sensitive spot for me personally, coming from a harmonic analysis group for my math PhD, but the short version basically goes like: Up until the 2010s or so, a huge portion of applied math was driven by results from "continuous math": functions mostly take values in some unbounded continuous vector space, they're infinite-dimensional objects supported on some continuous subset of R^n or C^n or whatever, and we reason about signal processing by proving results about existence, uniqueness, and optimality of solutions to certain optimization problems. The infinite-dimensional function spaces provide intellectual substance and challenge to the approach, while also being limited in applicability to circumstances amenable to the many assumptions one must typically make about a signal or sensing mechanism in order for the math model to apply.

This is all well and good, but it's a heavy price to pay for what is, essentially, an abstraction. There are no infinities, computed (not computable) functions are really just finite-dimensional vectors taking values in a bounded range, any relationships between domain elements are discrete.

In this circumstance, most of the traditional heavy-duty mathematical machinery for signal processing is irrelevant -- equation solutions trivially exist (or don't) and the actual meat of the problem is efficiently computing solutions (or approximate solutions). It's still quite technical and relies on advanced math, but a different sort from what is classically the "aesthetic" higher math approach. Famously, it also means far fewer proofs! At least as apply to real-world applications. The parameter spaces are so large, the optimization landscapes so complicated, that traditional methods don't offer much insight, though people continue to work on it. So now we're just concerned with entirely different problems requiring different tools.

Without any further context, that's what I would assume your colleague was referring to, as this is a well-understood mathematical dichotomy.

I made continuous/discrete distinction more in order to take a jab at people that don't know measure theory and therefore think these approaches can't be unified. (Though I do know for the record that in some cases, like the ones you mention, there is no overarching, unifying theory.)

Other than that I agree with you with everything up to the point where you say "Without any further context...".

The dichotomy that you describe needs graduate-level mathematics to be properly understood in the first place. I'm not sure why (luck?) it seems you are biased by being surrounded by people who are competent at ML and math. I guarantee you that is not the case. If you review for any of the big conferences (e.g. NeurIPS) you will see that really fast. :(

I don't think the undergrad math and statistics are really the problem. I think even if people understood the math, there would be some who questioned whether there's something epiphenomenon arising from the math, because there are more fundamental issues. Students in philosophy of mind classes were confronting these issues long before the DL boom started, with the Chinese Room argument. I.e. even if we accept as premise that there's a pre-described and deterministic set of steps that gives rise to seemingly intelligent behavior, we can be divided over what properties to ascribe to that system. Does it "understand"? Is it "thinking"? Is it a "person"? Why?

- we've never settled on a definition of consciousness

- we don't understand how biological brains create qualia

- we're not entirely in agreement over when human biological brains stop having ethical standing

- we disagree about how necessary embodiment or the material world outside the brain is to "mind"

- we disagree on what ethical standing to give to non-human animals with biological brains

Because we know so little, perhaps the only people that can believe strongly that current ML models are conscious have embraced panpsychism. But also because we know so little, including _what consciousness is_, I also don't think one can be confident that no ML model today has any consciousness whatsoever.

Recently I was re-reading some Douglas Hofstadter and in Metamagical Themas he has a dialogue about the "Careenium" in which on different (time)scales a physical system of balls and barriers appears to have qualitatively different kinds of dynamics. Hofstadter earnestly wants to use this to explain how "I" can direct or constrain in a top-down way some of the physical mechanisms that make me up while at the same time in a bottom-up way they comprise and animate me. I don't know if it was intentional, but it seems awfully similar to a beer-can apparatus that Searle somewhat dismissively uses as example of a potential computing substrate which could surely never support a "mind" no matter how it is programmed.

Two academics, both well respected, both who spent decades thinking about the mind and how it comes to be and how we should think about it, and both used an inscrutable complex physical (non-biological) apparatus to make points which are diametrically opposed about how the mind arises from more or less specific physical conditions.

I have in mind a property fwomp which I can't clearly define. But I have fwomp, and I'm pretty sure you have fwomp. I don't know where fwomp comes from, how it works or what are the necessary conditions for its persistence. From the preceding sentences, can you know with certainty that U-Nets don't have fwomp?

Well, you could write down a model for the full interaction between synapses and probably endow the whole thing with an RL-like environment so that you can model the interactions. The solutions of the model would then represent pretty exactly what a conscious being would do.

But it is hopeless to solve such a model, already writing it down would be a challenge.

There is some work in this direction to find a meaningful model by Eliasmith.

I think we're missing each other.

Whether you can write it down as a giant complex pile of more simple operations does not cut to the core issue: is consciousness a property of a computation which is present or not regardless of the physical substrate on which the computation happens, is consciousness a property of certain physical systems which isn't present even in extremely faithful computer simulations of those systems, or consciousness something else? (And how does ethical consideration of a system depend on consciousness?)

A computer simulation of weather doesn't have the actual wetness of rain, no matter how detailed the simulation is.

Addition can be present in a 4th-grader doing arithmetic, an abacus, an analogue computer or an electronic computer, even if they have different limitations on the size of numbers they can handle, at what speed, etc.

I'm going to ignore the 'full interaction between synapses' part b/c we don't know what kind of detail is needed for a computer model to capture the behavior of a conscious being. But the point is -- if you _had_ such a model, and could run it forward, people would still be divided as to whether it was actually conscious or not, because the basic properties of consciousness are not settled.

Ok, I see the point you wanted to make.

You ask "is consciousness a property of certain physical systems which isn't present even in extremely faithful computer simulations of those systems, or consciousness something else?" and later say "if you _had_ such a model, and could run it forward, people would still be divided as to whether it was actually conscious or not".

It seems to me that you thereby answer your first question with "no", as it seems that for you, by the last sentence, consciousness is a metaphysical property...

Which I would be fine with: I think we will only know more once we have an entity that looks and acts humanly (so there is reason to suppose consciousness might exist in it) and then try to understand its formal, inner workings and compare it to the human brain. No matter the outcome, it will be very interesting.

> It seems to me that you thereby answer your first question with "no", as it seems that for you, by the last sentence, consciousness is a metaphysical property...

That is absolutely not what I said. From my first post in this conversation I said "we've never settled on a definition of consciousness". In my preceding post, in describing a hypothetical, I said "people would still be divided as to whether it was actually conscious or not, _because the basic properties of consciousness are not settled_."

"People will disagree about X" does not in anyway amount to "I believe that X is a metaphysical property."

The absence of a clear definition means that people will continue to disagree regardless of the evidence about whichever physical system that you examine, because they're bringing different conceptions of consciousness to the table. If you know in advance that no evidence will be able to answer the question as currently framed, there's no need to wait for it; we must refine the question first.