[Column] Artificial Intelligence III

-
Recap :What are AI good at ? Classification !
  1. Two steps of classification : feature extract and feature classify
  2. What is image mean to computer ? pixel , matrix , resolution 
    1. Black and white , 0 - 255
    2. Colorful (0-255,0-255,0-255), RGB , tensor 
    3. Channel number (# dimension)
  3. Tensor 
    1. 0 order : scalar
    2. 1 order : vector
    3. 2 order : matrix
  4. Before deep learning (human designed feature)
    1. graphical feature is defined by computer vision,
      1. edge
      2. texture
    2. after combined with machine learning 
      1. object recognization
      2. object detection
  5. Convolution
    1. vector, matrix, tensor can be applied.
    2. steps of convolution : repeat (slide, extract, inner product)
    3. convolution kernel 
  6. histogram of oriented gradients (HOG) - to get the edge of figure 
    1. step 1 : using convolution calculation get the edge of the figure 
    2. step 2 : divide figure into multiple zones 
    3. step 3 : according to edge feature , amplitude, orientation conduct statistic. 
    4. step 4 : get histogram 
  7. Image net
Deep Neutral Network (DNN)
  1. Deep neutral network 
    1. automatically get the feature of the figure 
    2. before is feature extract > feature classify (two steps independently)
    3. after is  (feature extract and classify ) combined together
  2. What is dnn ?
    1. constructed by multiple layer 
    2. We can view each layer as a converter , from ( the path from realistic to abstract)
    3. just like learning English (letter > vocabulary > sentence ...)
    4. What is in dnn ?
      1. CONVOLUTIONAL LAYER
      2. FULL-CONNECTED LAYER
      3. SOFTMAX LAYER
      4. NON-LINEAR ACTIVATION LAYER (RELU)
      5. POOLING LAYER
  3. Convolution layer [1]
    1. the layer using convolution  calculation to convert feature or original data
    2. using multiple convolution kernel in one layer
    3. FEATURE MAP ~ 3 order tensor (final output of convolutional layer)
    4. the process from feature figure to feature vector
  4. Full connected layer [2]
    1.  inner product operation to combined all of the results into one vector as output.
    2. yk = X.Wk +bk ; Y  = sum (yks)
      1. X : vector ( as input)
      2. Wk : parameter vector 
  5. Softmax layer [n]
    1. the last layer of classification network
    2. output each figure's probability according to each class
  6. Non-linear activation layer 
    1. order : convolutional layer  > non-linear activation layer
    2. order : full connected layer   > non-linear activation layer
    3. the purpose of converting linear function to non-linear function is the avoid superposition effect on linear function that multiple layer effect is gone.
    4. convert LINEAR FUNCTION (such as convolutional layer and full connected layer ) into NON-LINEAR FUNCTION 
    5. commonly used non-linear function 
      1. logistic function 
      2. hyperbolic tangent function
      3. rectified linear function  (aka ReLU layer)
        1. ease to use and understand 
        2. fast converting
        3. good effect
  7. Pooling layer
    1. the propose of this layer : to decrease feature figure's resolution to decrease the computing amount
    2. where we add ? : commonly add this layer after multiple convolutional layer
    3. the operating ? :
      1. step 1 : dividing channel  to get multiple matrix 
      2. step 2 : for each matrix we divide it into multiple square, and the size of the square are the same.
      3. step 3 : for each square we pick up the max or average valued combined it into a  new matrix 
      4. step 4 : put all of the channels together in original order , and the 3 order tensor is done (aka pooling layer output)
    4. max pooling layer and average pooling layer
Biological Neural Networks
  1. the idea of DNN is come from biological neural networks
  2. the algorithm of DNN is reference from a type of biological neural networks
  3. till now , DNN cannot replace BNN, DNN is for specific task only.
DNN Training
  1. the concept of training is the process to get the "optimize parameters"
  2. for high order linear classification training : "BACK PROPAGATION" algorithm is announced. it is the most effective way to train the DNN.
    1. adjust methods include
      1. CHAIN RULE
      2. STOCHASTIC GRADIENT DESCENT
  3. What is deep means in DNN
    1. multiple layers , the number of the layers (the number of the parameters)
Its time for Deep Learning
  1. two major opportunities : data and hardware
    1. diversity huge amount of data, just like the fuel that you used to launch a rocket called deep learning 
    2. computing power, CPU to GPU
 Is multiple layer always good ?
  1. OVER-FITTING (good training data, bad testing data, there are always exceptions)
  2. UNDER-FITTING (bad talent , bad algorithm , even if the data is good , the parameter is good , but the core is bad, cause the overall performance has its limitations)
  3. the way to solve over-fitting and under-fitting  The solutions :!!!
    1. WEIGHT DECAY
    2. REGULARIZATION
  4.  multiple layer comes out GRADIENT VANISH and no direction (guide)
  5. the way to solve gradient vanish are
    1. BATCH NORMALIZATION
    2. SHORT-CUT 




Comments

Post a Comment