1. Difference between CNN and ANN [3]

•      Similarity between NN and CNN

CNN made up of neurons that have learnable weights and biases. Each neuron receives some inputs, performs a dot product and optionally follows it with a non-linearity. they still have a loss function on the last (fully-connected) layer. all the tips/tricks we developed for learning regular Neural Networks still apply.

·         So what changes?

Regular Neural Nets

NN Receive an input (a single vector), and transform it through a series of hidden layers. Each hidden layer is made up of a set of neurons, where each neuron is fully connected to all neurons in the previous layer, and where neurons in a single layer function completely independently and do not share any connections. The last fully-connected layer is called the “output layer” and in classification settings, it represents the class scores.

Regular Neural Nets don’t scale well to full images.

For example, images are only of size 32x32x3 (32 wide, 32 high, 3 color channels),  so a single fully-connected neuron in a first hidden layer of a regular Neural Network would have 32*32*3 = 3072 weights. This amount still seems manageable, but clearly this fully-connected structure does not scale to larger images.  For example, an image of a more respectable size, e.g. 200x200x3, would lead to neurons that have 200*200*3 = 120,000 weights. Moreover, we would almost certainly want to have several such neurons, so the parameters would add up quickly! Clearly, this full connectivity is wasteful and the huge number of parameters would quickly lead to overfitting.

3D volumes of neurons. CNN

unlike a regular Neural Network, the layers of a ConvNet have neurons arranged in 3 dimensions: width, height, depth. (Note that the word depth here refers to the third dimension of an activation volume, not to the depth of a full Neural Network, which can refer to the total number of layers in a network.)   For example, the input images are an input volume of activations, and the volume has dimensions 32x32x3 (width, height, depth respectively). As we will soon see, the neurons in a layer will only be connected to a small region of the layer before it, instead of all of the neurons in a fully-connected manner. Moreover, the final output layer would have dimensions 1x1x10, because by the end of the ConvNet architecture we will reduce the full image into a single vector of class scores, arranged along the depth dimension. A ConvNet arranges its neurons in three dimensions (width, height, depth), as visualized in one of the layers.  Every layer of a ConvNet transforms the 3D input volume to a 3D output volume of neuron activations. the red input layer holds the image, so its width and height would be the dimensions of the image, and the depth would be 3 (Red, Green, Blue channels.

2. Definition of CNN (Convulaiton of Neural Network )

A Convolutional Neural Network (CNN) is a deep learning algorithm that can recognize and classify features in images for computer vision. It is a multi-layer neural network designed to analyze visual inputs and perform tasks such as image classification, segmentation, and object detection, which can be useful for autonomous vehicles. CNN can also be used for deep learning applications in healthcare, such as medical imaging [2].

3. Inheritance from the real world [4]

The object recognition issue was inspired by examining the way our own visual cortex operates. Start with an input image then extract a few primitive features and combines those features from certain parts of the objects. Finally, it pulls together all of the various parts to form the object itself

It is a hierarchical way of seeing
Input image
Primitive features: First layer simple feature is detected
Object Parts
Then these are combined to shape more complicated features in the second layer
CNN was developed from this sequence of steps  

4-Application [4]

 Image processing, Handwritten text/digital recognition
Natural objects classification (Photos and Videos ). Allow it to extract the element from the image. It is not an easy task for compute. Segmentation, Face detection. Recommender systems. Speech recognition. Natural language processing

5. The original goal of CNN

8- Shallow Neural Network, why not [4]

First, let see why traditional shallow neural networks do not work as well as CNN.
Important steps in the modeling for classification with shallow NN is the feature extraction step.
These chosen features could be simple be the color, object edges, pixel location, or countless other features . The better and more effective the feature sets you to find a more accurate and efficient the image classification Selecting the best features is tremendously time –consuming and is often ineffective.

 Convolution neural network (CNNs)
CNN tries to solve this problem by using more hidden layers and also with more specific layers.  Instead of you choosing image features, to classify dogs vs. cat  for instance CNN can automatically find those features
 

9. Digital recognization [4]

Training and inference
Let consider the digit recognition problem, We would like to classify images of handwritten numbers, where the observation are the intensity of the pixels and the target will be a digit.

If we look at the pipeline of our DL process, we can see the following phases:
Pre-processing : First pre-processing of input data Convert the image to a readable and proper format for our network 
Training: Second, training the deep learning model Then, an untrained network is a feed with a big dataset  of the images and that is we build CNN
Inference and development: Third inference and deployment of the model Finally,  we use the trained model in the inference phase, which classifies a new image by inferring 

10. CNN  Layers [4]

Deep layer neural network  not only has multiple hidden layers, the type of layers and their connectivity also is different from a shallow NN, In that it usually has multiple convolution layers, pooling layers, as well as  fully connected layers

Convolution layers

Conv layers applied a convolution operation to the input and pass the result to the next layer.

Pooling layers

The pooling layer combines the outputs of a cluster of neurons in the previous layer into a single neuron in the next layer.

Fully connected layers

The fully connected layers connect every neuron in the previous layer to every neuron in the next layer.

What is pooling layer
Type of Pooling
max pooling and average pooling 
Hyperparameters
f: filter size
s: stride 

Parameter Sharing: A feature detector (such as a vertical edge detector) that is useful in one part of the image is probably useful in another part of the image [5]

The sparsity of connection: In each layer, each output value depends only on a small number of inputs 

What is computer vision [5]

it is a complicated task. you input picture in computer and computer tell you what is in this picture 

1 Data Augmentation [5]

Most computer vision task needs more data. Data augmentation is one of the techniques used to improve the performance of the computer vision system. it just mirroring the vertical axes. 

1.1 Random cropping [5]

you have given a dataset. let pic some take a few random crops and it produces a different example of your training set . it is a perfect example of data augmentation. The random crop is a subset of the actual image or dataset.  We can also use other techniques for example rotation, Sheering, Local warping for data augmentation  

1.2 Color shifting [5]

if you have given a picture and you add some RGB channel with different distortion. The value of RGB is drawing from a probability distribution. The motivation of this is that the sunlight is lit yellow. That could easily change the color of the image but the identity of the cat(Y) is the same, We used different algorithms for color shifting, for example, PCA or PCA color augmentation.

 2. State of Computer Vision

There few things which unique about the Deep learning application and computer vision about the state of computer vision. Today; we have a decent amount of data for speech recognization and image recognition. even there is a large amount of data for image recognition or image classification because image recognization is a very complicated problem and still we have needs more data. There is some problem like object detection where we have even fewer data.  image recognization is a problem where we looking in the picture and telling you is it a cat or not. where objected detection is a problem where we looking picture and putting boundary box around the object and telling you where is an object in the picture. if we have less data then we avoid using the simple ML classification algorithms or avoid less handling ( less carefully design features ) engineering. when we have lot of data then we used ANN. When we have less data then hand engineering is the best way to get good performance. When; we see ML there is two sources of knowledge, 1) label data 2) hand engineering 

References 

[1] A beginner guide of CNNA 1
https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/
[2] What is a Convolutional Neural Network 
https://missinglink.ai/guides/convolutional-neural-networks/convolutional-neural-network-architecture-forging-pathways-future/
[3] CS231  in Convolutional Neural Network for Visual Recognition
https://cs231n.github.io/convolutional-networks/
https://cs231n.github.io/understanding-cnn
[4] Deep Learning with TensorFlow
https://courses.cognitiveclass.ai/courses/course-v1:CognitiveClass+ML0120ENv2+2018/courseware/407a9f86565c44189740699636b4fb85/12eab34ec218468995e4d06566ef4a32/
[5] Deep Learning specialization 
https://www.coursera.org/specializations/deep-learning