Here are the slides from my PyCon IE 2013 keynote speach.
The code and resources used in the examples can be found here.
Monday, October 14, 2013
Saturday, September 29, 2012
Sunday, July 15, 2012
Big "A" Little "i": EuroPython slides
Here are the slides from my EuroPython 2012 talk. Now that I have developed these ideas into a talk, I am not sure whether or not I will go back to complete the series I started on this topic. There was some interest in turning this material into a short training course, that is appealing to me as I was not happy with how brief I had to be to fit the ideas into a 45 minute talk.
Sunday, October 09, 2011
PyCon Ireland Slides (Once more with feeling!)
This is largely the same as the slides I uploaded previously after PyCon UK, with some corrections and minor improvements.
Sunday, September 25, 2011
PyCon UK Slides (Is this thing on?)
Here are my slides from PyCon UK 2011. Next week after a rather long hiatus, I will resume the series on practical artificial intelligence.
Sunday, December 05, 2010
Bid "A" Little "i" - Part II
In Part I I gave what I hope was a convincing argument for why software developers should care about Ai. In this article I will show how looking at problems from an Ai perspective can yield immediate benefits. I will then introduce the fundamental tenet of Ai, the red thread that ties all the approaches I will discuss (and indeed those I will not) together.
It is Good to Talk
A seemingly obvious but rather useful thing that Ai does for us as software developers is it equips us with ways of thinking about thought and ways of talking about thought. By giving the techniques and algorithms we use names, it makes it easier to reason about them and to discuss them. The discussion or communication aspect should be familiar to most software developers as it is widely considered one the major benefits of Design Patterns, it is similarly beneficial in Ai.
An example of a rather simple algorithm that has been given a name in Ai is: Generate and Test. Lets say your task is to recognise a specific make and models of cars given a set of features, what would you do? Well, you could build yourself the Rube Goldberg machine below.
So what does this device do? The Generator flips through all the pages of the popular car buyers and sellers magazine Autotrader, it offers the cars on each page as possibilities to the Tester. The Tester takes each possibility from the Generator and checks its features to determine if it matches the type of car being searched for. Cars that don't match are discarded and the Tester moves on to the next possibility, once a match is found it presented as the result of the computation, if all the pages are exhausted but still no match is found no result is returned.
Why glorify such as simple computation with a name? As we eluded to earlier it allows us to answer concisely questions like: "How are you going to solve this problem?". We can now answer: "I am going to solve this problem using Generate and Test". Even more importantly we can now talk precisely about the properties of the Generator and the Tester.
Interesting properties of the Tester include:
Framing Problems
One of the most important things human beings do in problem solving is framing questions in a way that makes them easier to solve. We are all aware of this intuitively from phrases like: "looking at it from another point of view I realised...". My thesis is that the ability to frame problems appropriately is the basis for intelligence and indeed genius. Problem framing is at the heart of most of the topics that I am going to cover over the next few posts. I would argue that one of the biggest problems for Artificial General Intelligence (which as you may remember from Part I is about artificial agents comparable to, or smarter than humans) is that our algorithms are not very good at framing problems for themselves. A human being has to frame the problem then the computer can do its thing. This limitation places an upper bound on how intelligent an artificial agent can be.
This limitation is however perfectly acceptable for the masquerade of intelligence that I am interested in here. For our purposes we need to make the concept of problem framing more concrete, hence heretofore I will refer to Representations instead. To show how choosing the right representation is key to solving problems in general and in particular to conditioning problems for solution by a computer another example is order.
Most children have seen the puzzle of the farmer, the fox, the goose and the grain. All four are on one side of a river and the farmer wants to get all of them to the other side of the river (I always wondered why the farmer was ferrying a fox around). However the fox if left alone with the goose will eat it, the goose if left alone with the grain will eat it. A child would probably solve the problem relatively quickly by trial and error. What we need to do though is condition the problem for solving by a computer, we need a representation:
This representation looks promising as it makes explicit which side of the river everyone is on, lets see if we can use it to make progress with the puzzle. We did not mention it earlier but to get across the river the farmer must take the other across in a boat that has only room for two. What happens if our robot farmer takes whatever happens to be nearest to him across the river, in this case that is the fox:
This didn't work out too well, the goose was left unattended to feast on the grain - game over, we lost. Even though we lost the game, the representation has exposed an important constraint that might be obvious to us humans but must be explicitly stated for an Ai farmer, things that eat each other cannot be on the same side of the river without the farmer also being there. Our representation is serving us well so far, it has exposed constraints that we can exploit to solve the problem. The constraint shows which state transitions are valid and which ones are not:
There is really not much left for us to do to fully program our robot farmer. The problem can now be solved as a series of valid transitions between states involving the farmer moving from one side of the river to another. Once an appropriate representation has been chosen it is often a mechanical process to get to the solution, standard computer science algorithms (searching, sorting, finite state machines an so on) exploiting the constraints exposed are all that is required to take us the rest of the way.
It may seem like overkill to solve this rather simple puzzle which could be expressed in a few "if-else" statements in the manner we have but it is not! Take another look at what we have done, we have taken the puzzle and generalized it into a good representation and the constraints the representation exposes. This means we can solve any puzzle of this nature without any change to the code. Even if we change the entire cast, just by adding the knowledge of who eats who, that new puzzle is automatically solved.
An even more important point to note is that the person programming the farmer robot does not need to be able to solve the puzzle. They solve the problem at a different level, I doubt for example that any of the engineers that worked on Deep Blue were chess grandmasters. This is not such a big win for a simple puzzle like this, however there are problems which human beings cannot solve because of the time it would take, the errors a human would introduce, the amounts of data that need to be processed or host of other reasons. Hence Ai gives us the ability to solve problems that would otherwise be impossible to tackle!
We have seen how to get some quick wins out of Ai thinking and introduced the central concept of representations (as well as the ancillary concept of constraints). Having set the scene in Part I and given a technical introduction to Ai in this article the coming instalments will cover the main classical Ai techniques.
It is Good to Talk
A seemingly obvious but rather useful thing that Ai does for us as software developers is it equips us with ways of thinking about thought and ways of talking about thought. By giving the techniques and algorithms we use names, it makes it easier to reason about them and to discuss them. The discussion or communication aspect should be familiar to most software developers as it is widely considered one the major benefits of Design Patterns, it is similarly beneficial in Ai.
An example of a rather simple algorithm that has been given a name in Ai is: Generate and Test. Lets say your task is to recognise a specific make and models of cars given a set of features, what would you do? Well, you could build yourself the Rube Goldberg machine below.
So what does this device do? The Generator flips through all the pages of the popular car buyers and sellers magazine Autotrader, it offers the cars on each page as possibilities to the Tester. The Tester takes each possibility from the Generator and checks its features to determine if it matches the type of car being searched for. Cars that don't match are discarded and the Tester moves on to the next possibility, once a match is found it presented as the result of the computation, if all the pages are exhausted but still no match is found no result is returned.
Why glorify such as simple computation with a name? As we eluded to earlier it allows us to answer concisely questions like: "How are you going to solve this problem?". We can now answer: "I am going to solve this problem using Generate and Test". Even more importantly we can now talk precisely about the properties of the Generator and the Tester.
Interesting properties of the Tester include:
- Accuracy: Do we require 100% accuracy or would 99% accuracy suffice? A Bayesian mathematician would tell you that 99% accuracy is is definitely not good enough!
- Speed: A tester that contemplates each possibility for too long may not find a solution in a reasonable amount of time.
- Complete: If there exists a car in the world we would like our generator to produce it. By this measure our choice of using the Autotrader magazine might have been a bad one since it is unlikely that all possible cars types are always for sale in this magazine.
- Inform-able: If I am looking for a small hatchback it does not make sense to offer a van as a possibility to the tester, it would be nice if we could tell the generator that.
- Non-Redundant: We do not want the same car offered as a possibility more than once, as could happen if our page turner went through pages at random rather than in sequence.
Framing Problems
One of the most important things human beings do in problem solving is framing questions in a way that makes them easier to solve. We are all aware of this intuitively from phrases like: "looking at it from another point of view I realised...". My thesis is that the ability to frame problems appropriately is the basis for intelligence and indeed genius. Problem framing is at the heart of most of the topics that I am going to cover over the next few posts. I would argue that one of the biggest problems for Artificial General Intelligence (which as you may remember from Part I is about artificial agents comparable to, or smarter than humans) is that our algorithms are not very good at framing problems for themselves. A human being has to frame the problem then the computer can do its thing. This limitation places an upper bound on how intelligent an artificial agent can be.
This limitation is however perfectly acceptable for the masquerade of intelligence that I am interested in here. For our purposes we need to make the concept of problem framing more concrete, hence heretofore I will refer to Representations instead. To show how choosing the right representation is key to solving problems in general and in particular to conditioning problems for solution by a computer another example is order.
Most children have seen the puzzle of the farmer, the fox, the goose and the grain. All four are on one side of a river and the farmer wants to get all of them to the other side of the river (I always wondered why the farmer was ferrying a fox around). However the fox if left alone with the goose will eat it, the goose if left alone with the grain will eat it. A child would probably solve the problem relatively quickly by trial and error. What we need to do though is condition the problem for solving by a computer, we need a representation:
This representation looks promising as it makes explicit which side of the river everyone is on, lets see if we can use it to make progress with the puzzle. We did not mention it earlier but to get across the river the farmer must take the other across in a boat that has only room for two. What happens if our robot farmer takes whatever happens to be nearest to him across the river, in this case that is the fox:
This didn't work out too well, the goose was left unattended to feast on the grain - game over, we lost. Even though we lost the game, the representation has exposed an important constraint that might be obvious to us humans but must be explicitly stated for an Ai farmer, things that eat each other cannot be on the same side of the river without the farmer also being there. Our representation is serving us well so far, it has exposed constraints that we can exploit to solve the problem. The constraint shows which state transitions are valid and which ones are not:
It may seem like overkill to solve this rather simple puzzle which could be expressed in a few "if-else" statements in the manner we have but it is not! Take another look at what we have done, we have taken the puzzle and generalized it into a good representation and the constraints the representation exposes. This means we can solve any puzzle of this nature without any change to the code. Even if we change the entire cast, just by adding the knowledge of who eats who, that new puzzle is automatically solved.
An even more important point to note is that the person programming the farmer robot does not need to be able to solve the puzzle. They solve the problem at a different level, I doubt for example that any of the engineers that worked on Deep Blue were chess grandmasters. This is not such a big win for a simple puzzle like this, however there are problems which human beings cannot solve because of the time it would take, the errors a human would introduce, the amounts of data that need to be processed or host of other reasons. Hence Ai gives us the ability to solve problems that would otherwise be impossible to tackle!
We have seen how to get some quick wins out of Ai thinking and introduced the central concept of representations (as well as the ancillary concept of constraints). Having set the scene in Part I and given a technical introduction to Ai in this article the coming instalments will cover the main classical Ai techniques.
Sunday, September 26, 2010
Big "A", Little "i" - Part I
Back in the 1980's there was a bubble in the technology industry that was to prove a good model for the dot-com bubble that would follow a couple of decades later. The 1980's bubble was due to the excitement and promise of Artificial Intelligence (AI). Thanks to some rather impressive but narrowly focused programs there was a feeling that the age of thinking machines was upon us. Investors and technologists alike rushed to get in on the act, a lot of money was spent, made and lost. People thought that AI technology would change the world1, but when it didn't change the world overnight much the interest faded away. Of course it did not help that in the excitement a lot of money was spent on functional but rather basic software, like rudimentary Expert System shells.
The parallels with the dot-com bubble are obvious, a new technology comes along that seems like it will change the way we live, work and play, everyone gets overexcited, but it all comes to naught. Except... that it is not true! We know the rest of the story, the boom-bust cycle was just a capitol markets phenomenon and belies the fact there was real revolution taking place. Today we have internet companies that have market capitalisations in the tens of billions so clearly there was value there. Furthermore that is just the direct value to companies whose businesses are focused on the internet. There another fundamental shift from a technology perspective that is less conspicuous but more pervasive. Most software today has to be internet enabled, it is either entirely web-based or connects to web-services. Users now expect the richer experience enabled by the web, applications that sit entirely on the user's desktop with no gateway to the outside world seem dated and quaint.
The web is so integral that almost every working programmer must be at least aware of the technologies underlying it. I would argue that the same can be said about AI, though its importance is a little more subtle. The AI in most systems tends to be obscured, partly because it is not directly visible from the user interface, partly due to its nature2 and partly because usually it does not make up a large part of the system. However a lot of the worlds most interesting software is only possible because of AI. AI systems are in use today in fields such as: flight control, manufacturing robots, search engines, fraud detection, medical diagnosis and games to name but a few. It is like raisins in a loaf of Raisin Bread - they may not occupy a large part of the volume of the bread, but you cannot have Raisin Bread without raisins!
Now you say to me: but AI has failed to make progress over the past few decades, where are those thinking machines and robots, show me the robots! Now we are getting to the crux of the matter, for AI can be about understanding and emulating human intelligence but it does not have to be! Indeed my thesis is that there are two closely related3 but nevertheless distinct sides to AI, the Science of AI and the Business of AI.
From the science perspective of trying to understand and emulate human cognition there may seem to be very little progress being made, but that is a misconception4 which unfortunately I do not have time to discuss in detail here. At any rate this aspect of AI is often referred to as Artificial General Intelligence, but I like to think of it as aI little "a", big "I". That is because there is not that much that is artificial about this endeavour, the ultimate goal is to understand cognition to the point where we can create an artificial agent that thinks at human level and beyond.
The business of AI is what I think people interested in creating smart applications should be aware of. In contrast to the science of aI, the goal here is to build applications and machines that appear to be intelligent. Since in this case we only care about intelligence in the context of the specific application being built, this type of AI is typically referred to as Narrow Artificial Intelligence. I like to think of it however as Ai, big "A", little "i". Since the applications and machines only need to appear intelligent, so it is definitely artificial. Also the means by which this is achieved is not that important, a cheap trick that works is as worthy as an actual intelligent agent. I hope I did not give you the impression that I am down on Ai, quite the opposite! The techniques of AI used in narrow contexts are what enable the applications I mentioned earlier and many more besides. Without an understanding of basic AI algorithms, you may find yourself spending a lot of time creating unreliable and hard to maintain programs, that could be expressed in much simpler ways with standard AI techniques.
OK, so that was a very long-winded way to say that working programmers should know something about AI. After all that there had better be a pay-off and there is. The next few instalments will be introductions to what I consider the most useful and generally applicable Ai techniques. Stuff that you are guaranteed to find useful because it has been useful thousands of times in the past. Stay tuned...
1. And indeed it has, though perhaps not in the way people imagined back then!
2. When you meet a really smart person you might exclaim "Wow! That girl is really smart", I doubt you have ever thought "Wow! That girl has a smart brain".
3. Especially in terms of the techniques used, the research conducted and indeed the personnel involved.
4. This is an argument better made by the people working on Artificial General Intelligence: http://singinst.org; http://www.idsia.ch
The parallels with the dot-com bubble are obvious, a new technology comes along that seems like it will change the way we live, work and play, everyone gets overexcited, but it all comes to naught. Except... that it is not true! We know the rest of the story, the boom-bust cycle was just a capitol markets phenomenon and belies the fact there was real revolution taking place. Today we have internet companies that have market capitalisations in the tens of billions so clearly there was value there. Furthermore that is just the direct value to companies whose businesses are focused on the internet. There another fundamental shift from a technology perspective that is less conspicuous but more pervasive. Most software today has to be internet enabled, it is either entirely web-based or connects to web-services. Users now expect the richer experience enabled by the web, applications that sit entirely on the user's desktop with no gateway to the outside world seem dated and quaint.
The web is so integral that almost every working programmer must be at least aware of the technologies underlying it. I would argue that the same can be said about AI, though its importance is a little more subtle. The AI in most systems tends to be obscured, partly because it is not directly visible from the user interface, partly due to its nature2 and partly because usually it does not make up a large part of the system. However a lot of the worlds most interesting software is only possible because of AI. AI systems are in use today in fields such as: flight control, manufacturing robots, search engines, fraud detection, medical diagnosis and games to name but a few. It is like raisins in a loaf of Raisin Bread - they may not occupy a large part of the volume of the bread, but you cannot have Raisin Bread without raisins!
Now you say to me: but AI has failed to make progress over the past few decades, where are those thinking machines and robots, show me the robots! Now we are getting to the crux of the matter, for AI can be about understanding and emulating human intelligence but it does not have to be! Indeed my thesis is that there are two closely related3 but nevertheless distinct sides to AI, the Science of AI and the Business of AI.
From the science perspective of trying to understand and emulate human cognition there may seem to be very little progress being made, but that is a misconception4 which unfortunately I do not have time to discuss in detail here. At any rate this aspect of AI is often referred to as Artificial General Intelligence, but I like to think of it as aI little "a", big "I". That is because there is not that much that is artificial about this endeavour, the ultimate goal is to understand cognition to the point where we can create an artificial agent that thinks at human level and beyond.
The business of AI is what I think people interested in creating smart applications should be aware of. In contrast to the science of aI, the goal here is to build applications and machines that appear to be intelligent. Since in this case we only care about intelligence in the context of the specific application being built, this type of AI is typically referred to as Narrow Artificial Intelligence. I like to think of it however as Ai, big "A", little "i". Since the applications and machines only need to appear intelligent, so it is definitely artificial. Also the means by which this is achieved is not that important, a cheap trick that works is as worthy as an actual intelligent agent. I hope I did not give you the impression that I am down on Ai, quite the opposite! The techniques of AI used in narrow contexts are what enable the applications I mentioned earlier and many more besides. Without an understanding of basic AI algorithms, you may find yourself spending a lot of time creating unreliable and hard to maintain programs, that could be expressed in much simpler ways with standard AI techniques.
OK, so that was a very long-winded way to say that working programmers should know something about AI. After all that there had better be a pay-off and there is. The next few instalments will be introductions to what I consider the most useful and generally applicable Ai techniques. Stuff that you are guaranteed to find useful because it has been useful thousands of times in the past. Stay tuned...
1. And indeed it has, though perhaps not in the way people imagined back then!
2. When you meet a really smart person you might exclaim "Wow! That girl is really smart", I doubt you have ever thought "Wow! That girl has a smart brain".
3. Especially in terms of the techniques used, the research conducted and indeed the personnel involved.
4. This is an argument better made by the people working on Artificial General Intelligence: http://singinst.org; http://www.idsia.ch
Subscribe to:
Posts (Atom)