Newbie question: I've heard that PGMs are a superset of neural networks. In PGM materials that's I've read, the topology of the networks shown as example is made of node that are manually chosen and represent concept (smart student, good grades, difficult subject, etc.). Whereas a neural network example is usually a huge set of nodes that end up finding their meaning on their own. I also vaguely recall a tutorial in which you can highlight the nodes that contributed to the classification - the only thing is that they don't have meaning for a human. Then when the article states:
> restrict the way nodes in a neural network consider things to ‘concepts’ like colour and shapes and textures.
Aren't these just PGMs? Are they NNs? Is it just a methodology approach on how to select the topology? Don't you lose the automatic meaning / structure search? I'm a little bit confused...
PGMs are interesting in how they represent distributions over the values of their nodes. In neural networks, (for the most part) those nodes are deterministic, so from a PGM perspective the distribution is trivial (up until the final output prediction). Performing inference in a neural net with stochastic nodes would be crazy hard, so the best you can do is usually MC with some kind of reparametrization trick to keep your gradients around.
PGMs and neural networks are usually two completely different things. Neural networks involve graphs, but those graphs usually represent continuous relaxations of circuits -- rather than probability models.
I think the author is overstating the importance of being able to explain in human terms decisions made by a neural network. For instance , there is no one reason that I am able to recognize a dog as such. Any feature or combination of features I can think of can be had in another animal. Something deeper is happening when I am able to correctly identify dogs that is unexplainable, at least by me.
The examples normally given for wildly inaccurate predictions were concocted by training a separate neural network to trick the original neural network which seems be just showcasing the effectiveness of neural networks rather than highlighting a weakness.
Also, I would note that human intuition is not immune to tricks. For instance optical illusions regularly trick our perception.
No, the importance is massive, and you understate it. While we may all want to just believe, the regulatory ecosystem worldwide (where it applies, esp. in financial realm) demands that we provide explanations of why models made certain decisions on certain data. Without this ability, models will not be allowed to drive innovation or decisions in many areas of life, from financial (credit and risk) to medical (recommendations for treatment) to legal (best contract approach or best defense approach for a lawsuit).
Saying that humans make mistakes and cannot explain their decisions is, in fact, one of the very reasons we want to have better models. We hope they will do better than most people to create a better world. And their explanations will hopefully provide insight into how we as people make good (and bad) decisions.
And sure, we can change the laws over time, but having advanced models which can allow humans to understand decisions and even provide diagnostics to improve the models will be transformative. Until then, we will see massive impact in some areas of our lives, and frustrating holdbacks in others, driven either by the distraction of building for regulatory constraints or by choosing not to build in regulated areas at all.
We know why these models make decisions on data. They're optimizing for lower error rates.
The black box unveiled for a convolutional neural network is this: over the course of several thousand rounds of performing the dot product of the RGB values of a given training datum against a weight vector, this network has determined that a weight vector containing these values optimally reduce the error produced by "squashing" the output matrices of said dot products in a softmax function when the "squashed" value is compared against the pre-determined true value.
If you would also like to be able to correctly predict whether a given input is part of the class for which this model was optimized for, we suggest your weight vectors also contain these values as this will reduce the number of false positives and false negatives you will produce from your prediction.
No. If the machine is telling me what medicine to administer to the patient, then I want to know exactly what data points the machine thought were relevant (i.e., the relevant symptoms). Furthermore, I want to know what about those symptoms indicated to the machine a particular diagnosis.
The criticism of artificial intelligence in making decisions in lieu of human judgement often pits the process against an impossible ideal that doesn't exist and likely could not exist.
For instance, some research suggests that judges are a lot less likely to grant parole before lunch. The theory is that no parole would be the default no-brainer decision a judge can make, which makes sense. [0]
The fact is that many decisions are made by humans with, at best slight personal bias and at worst malice.
With a systematic AI approach you can be somewhat confident that the decisions made are based on a wide range of experience, optimizing for an agreed upon objective (e.g. likelihood of reoffense). It's not perfect but it's better and can improve a lot easier without the need to villainize individuals
>Something deeper is happening when I am able to correctly identify dogs that is unexplainable, at least by me.
There is an interesting experiment you can do to directly demonstrate this to yourself
From Julian Jaynes' The Origin of Consciousness in the Breakdown of the Bicameral Mind
> Take any two unequal objects, such as a pen and pencil or two unequally filled glasses of water, and place them on the desk in front of you. Then, partly closing your eyes to increase your attention to the task, pick up each one with the thumb and forefinger and judge which is heavier. Now introspect on everything you are doing. You will find yourself conscious of the feel of the objects against the skin of your fingers, conscious of the slight downward pressure as you feel the weight of each, conscious of any protuberances on the sides of the objects, and so forth. And now the actual judging of which is heavier. Where is that? Lo! the very act of judgment that one object is heavier than the other is not conscious. It is somehow just given to you by your nervous system. If we call that process of judgment thinking, we are finding that such thinking is not conscious at all. A simple experiment, yes, but extremely important. It demolishes at once the entire tradition that such thought processes are the structure of the conscious mind.
The DARPA proposal the authors are working on is driven by the military. Consider the applications of an AI whose job it is to find hidden weapons caches from aerial photography of a city, or likely location for IEDs along a road.
You could train a NN to do either, given lots of training data, but before risking the lives of special forces, and potential for collateral damage, you might want to question the AI as to why they think what they found was a hidden cache of weapons or roadside IED.
I agree the statement "humans are surprisingly good at explaining their decisions" is dubious at best. Most explanations are a matter of what's called reconstructive cognition, which is far from accurate in explaining the reasons for an action.
>For instance , there is no one reason that I am able to recognize a dog as such. Any feature or combination of features I can think of can be had in another animal. Something deeper is happening when I am able to correctly identify dogs that is unexplainable, at least by me.
Well, your categorization judgement is probabilistic. Some features make "dogness" more likely, some less likely. However, there is in fact a specific causal structure which constitutes dogness (ie: organism grown via natural reproduction along the canid species-history, with the accompanying genotype and within normal phenotypical variation). Your brain tries to learn a causal model which approximates the correct causal structure, given the evidence available to your senses.
Better: the human brain also seems to perform second-order inference, reasoning about which models predict most precisely, what's relevant to what, and which latent parameters in the learned causal model are most relevant.
So when you see a dog-like tail versus a cat-like tail, you can identify which apparent features are making you say "dog" or "cat" most strongly, which ones are the strongest evidence.
Explanation is hugely important, and the lack of it was one of the factors that killed NNs in the 90s. The ability to explain was also one of the reasons rule-based systems succeeded as well as they did, despite having huge disadvantages vs NNs for many problem domains.
AI for high-consequence decision-making is going to need the automatic feature extraction and robustness of NNs coupled with the explanatory transparency of expert systems. That's been the holy grail for almost 40 years.
Part of it is they want to know that the AI isn't discriminating in illegal ways (eg using race as a feature).
I also worked with the data science guys at a telecom in a past startup and they were so used to using white-box models that the thought of not having a decision tree or other means of explaining how the results came about scared them, just because they weren't used to it and they could check it for correctness etc. Not having that was scary to them.
Another way to think of it is, science vs. magic/customs. Indeed empirical knowledge can provide reasonable answers (e.g. when rubbing a particular mold over your injury it will heal quicker). On the other hand, understanding what an antibiotic is, how to properly dose it and potentially synthesize it from other sources, etc. is a far better tool for the job.
I suspect reality is a messy mix of science and magic/customs, and perhaps one way in which AI can help us is to figure out - within limits -
where and when to take the shortcuts (magic/customs) and when/where not to.
But of course, there are bound to be all sorts of weird unintended consequences when we let what is essentially an alien life-form come up with these things.
I think the gorilla/black people issue is a perfect example of this. An 'AI' might group gorillas and black people together because its 'magic' made it do so. On the one hand it might seem like a rather unfortunate 'mistake', but it's also not difficult to understand why this would happen. I've had more than one experience with white children causing embarrassment by comparing black people to gorillas/monkeys.
I'm not going anywhere in particular with this other than that 1) I don't think science and magic/customs are quite as easily disentangled in daily life, and 2) I have high hopes for the role of AI in all aspects of human development, but I expect it to cause all kind of issues that, in hindsight, we might write about in the same way we write about culture clashes or hypothetical alien visitations. Until we create AI that simulates or experiences actual human lifespans, they'll be basically aliens to us in all kinds of ways, small and big.
This seems flippant. If a car is fast we generally understand why. We don't need to worry that under some rarely-encountered combination of circumstances it will unexpectedly do a handbrake turn and open the fuel cap.
Speed limits on public roads place legal limits on the operation of automobiles. Rules in formula racing place limits on the speed which race cars can travel.a
There's a hell of a lot of money to be made by the person who cracks this. The major blockers preventing a lot of AI being rolled out across the EU are laws which stipulate that you have to be able to explain a decision to, for example, refuse a person credit.
Not to mention the fact that we can correct faulty assumptions on the fly if we can get the networks to introspect.
Meh.
Why not just use a decision tree for the credit decision? Perfectly explainable, and the feature domain isn't as difficult (and full of symmetry) as vision or language problems.
One issue I don't see considered is - how to ensure that explainable artificial intelligence doesn't lie? Right now, it may not be an issue, but as AI systems get complex ("smart") enough, one need to be sure that the introspective output isn't crafted to influence people looking at it.
Right now it looks like it's being used more as a "debugging" output to make more intelligent Al's. Once they can lie, we will have achieved that goal...
Let's say this is possible. How would we know that it (the AI) isn't doing a post-hoc rationalization, or just outright lying about its reasoning?
In other words, why do we trust humans more than machines? In fact, why do we not think of humans as machines - just ones made out of different materials? Why do we have this bias that machines are and must-be deterministic, and since humans aren't, they must not be machines? Furthermore, since we know that these AI models are sometimes stochastic, why do we still insist that they be explainable; when humans exhibit the same kind of output, we don't insist upon their determinism...?
I'm not certain that we can make these models - especially complex deep-learning CNNs and others like them - explainable, any more than an individual can tell you how his or her brain came up with the solution; most of the time, we employ post-hoc reasoning to explain our decisions, depending on how the output resolves. That - or we lie. Rarely do we say "I don't know" - because to do so is to admit a form of failure. Not admitting such is what helps religion continue, because when we don't know, we can ascribe the reason to another external force instead. If we would just be willing to say "I don't know - but let's try to find out" (insert XKCD here), we might be better off as a species.
I don't think an AI model will be any different - or can be. If we insist on having an AI be able to deterministically and truthfully tell us exactly how it arrived at such a conclusion, we must be ready to accept that we should do the same with human reasoning as well. Anything less would be hypocritical at best.
> restrict the way nodes in a neural network consider things to ‘concepts’ like colour and shapes and textures.
Aren't these just PGMs? Are they NNs? Is it just a methodology approach on how to select the topology? Don't you lose the automatic meaning / structure search? I'm a little bit confused...