[Column] Artificial Intelligence I & II

AI Development History

1. 1950 : Alan Turing , Thesis - Computing Machinery and Intelligence , Turning Test
2. 1951 : Marvin Minsky, Princeton Master Student, SNARC (1969 Turning Award)
3. 1955 : Allen Newell/ Herbert Simon/ Cliff Shaw, Logic Theorist, Reasoning   
4. 1956 : John McCarthy/ Claude Shannon/ Nathan Rochester, AI Artificial Intelligence
5. First Period (1956-1974)
1. 1963 : APPA, Project MAC, MIT CSAIL
2. 1964-1966, Joseph Weizenbaum, ELIZA, MIT
3. 1967-1972, Human Robot. Japan
6. Second Period (1980-1987)
1. 1960s, Edward Feigenbaum, expert system
2. 1970s, Stanford, MYCIN
3. 1980s, CMU,DEC,XCON,expert system
1. Expert system
2. Artificial neural network
4. 1982, John Hopfield, Hopfield net, Associative memory
5. 1986, David Rumelhart/ Geoffrey Hinton/ Ronald Williams, Thesis-Learning representations by back-propagating errors, prove backpropagation by experiment
6. 1982,Japan, 5g calculator, over ten years investment, the project finally fail.
7. Third Period (2011-Now)
1. Math Based and Solution Based Research - 
1. Statistical Learning Theory
2. Support Vector Machine
3. Probabilistic Graphical Model
2. Big Data and Powerful Computing Technology Comes out (Example , NVIDIA Tesla V100 is better than the fastest super computer in the world in 2001)
3. 2012, ILSVRC ( Image Net), Alex Net
4. 2016,Google, AlphaGo

AI Application and Marketing

1. Security Video Camera , Action Prediction, Vehicle and Person Recognizance,Traffic Flow 
2. Medical X-Ray Detect, Multiple Photos to 3D modeling human organ
3. Customer Service
4. Autopilot
1. 2004,DARPA,Mojave Desert Test Run for 142 miles, fail
2. 2010-2012 ,Google , Mojave Deseret, Accumulate 300 thousands miles without any accident
3. 2014, Baidu and BMW start to research autopilot.
5. Industrial Manufacturing 
1. Quality Assurance Automative
2. 2011 Hannover Messe, Industrial 4.0 (Sensors > Big Data > Valuable Messages ), Siemens and GE start to use it to predict risk and reduce material waste and energy consumption, increase production rate. 

What is AI 

1. Using machine to stimulate human beings (make a prediction for given input)
1. Face recognize
2. Voice recognize
3. Medical Diagnoise
4. Recommendation by analyse Customer's Buying Record 
5. Finance , Investment

What is the challenge of applying AI

1.  If the scenario is complicated 
1. It is hard to write the  compact rules to defined all of the possibilities.
2. Some of the input is hard to defined (no rules or regular pattern) 

What is the relationship between AI and Machine Learning

1. When AI predict things by learning , the method to realize it is called machine learning, (Not the only one but the main one)
2. Data > Analyse comes out regular pattern > Prediction 
3. Methods of machine learning 
1. Learn form Data
1. Supervised Learning
1. sample
2. Need Qualified Data from Human (and it takes time)
2. Unsupervised Learning
3. Semi-Supervised Learning
2. Learn from Action
1. Reinforcement Learning (Policy Gaining to Guide Action )
1. Not like up three cases , it does not need lots of  input and predict samples, it learns from action
2. Include (State,Action,Agent,Environment,Reward)
3. This is what AlphaGo used

AI Application I : Classification Introduction

1. What are the categories of classification 
1. Figures
2. Voices
3. Videos
4. Text
5. Others
2. Process of Classification
1. Input 
2. Extract Features
3. Classifier
4. Output
3. What is a good classifier ?
1. based on good features extraction (difference > features > good features)
2. For Figures : 方向梯度直方圖
3. For Voices : 梅爾頻率倒譜係數
4. For Videos : 光流直方圖
5. For Text : 詞頻率-逆文檔頻率
4. Classifier Design Basic -Vectors
1. feature vector
2. feature point
3. feature space
4. distance (adjust the degree of similarity)
5. Dot Product 
6. Cross Product
7. Classify function 
5. Types of Classifier
1. linear classifier : f(x1,x2,x3...,xn) = a1x1 +a2x2+...anxn
1. n is dimension 
2. a1,a2...an is parameters
6. Defined Classifier 
1. Feature vector x1,x2...xn > Classifier g(x) > Class y
2. What AI do is to get the function g(x)
7. How to train classifier ?
1. training(learning) > testing(test) > application(problem solving)
2. The process of training is the process to get the appropriate parameters.
3. What are the materials that are used to train the classifier > Data (Training Data)
8. Types of Data
1. Training Data (Need to know their class in advanced marked by human)
2. Testing Data (Need to know their class in advanced marked by human)
3. The process that human marked the data is called "annotation"
1. The process of annotation is time wasted 
2. The process of annotation needs related domain's expert
4. Training Set
1. A great classifier comes out with a great algorithm, that is a core issue in Machine learning.
9. Training Methods (Parameters Get)
1. Methods One : Perception
1. aka linear classifier algorithm 
2. principle : using mistake examples to train the algorithm
3. methods : "perception learning algorithm"
1. select initial parameters
2. Select a set of data from training set
3. If the data is in the wrong class , then replace the parameter by a modified parameter 
4. Repeat the steps above, till all of the training set are set in the right place 
4. Learning rate : the degree that the parameter revised 
5. What is the rule of revising parameters ?
1. Loss Function
2. Methods Two : SVM (Support Vector Machine)
1. Classification Margin
2. Support Vector
10. Classification Accuracy
11. Binary Classification
12. Multi-class Classification
13. Softmax

Summary 

• Classification can be realized in AI domain. 
• The two significant parts in classification are "feature" and "classifier"
• Feature : present in feature vector.
• Classifier is a function that convert feature vector to class.
• Steps of classifications :
• 1. feature extraction : the process from data to feature vector
• 2. Classifier Training  : train by training data and algorithm
• 3. Classifier Testing and Application
• Different Training algorithm will produce different classifier even the training data are the same.
• For training algorithm , Perception and SVM are the most known algorithms.
• They have their own Loss Function 
• Loss function can measure the difference between training data and output results.