The linear model is expressed by the current combination of features “ result - Feature set ” The correspondence between . Because of the limited expressive power of linear model , In practice , Only by adding “ Feature calculation ” To optimize the model . such as , In the advertisement CTR In application , except “ Title Length 、 Description length 、 Position 、 advertisement id,cookie“ And so on , And there's a lot of combinatorial features （ such as ” Position -cookie“ It indicates the user's preference for bits ）. in fact , Now many search engine advertising systems use Logistic Regression Model （ linear ）, And one of the most important jobs of the model team is “ Feature Engineering (feature engineering)”.

The idea of a linear model is “ Simple model + Complex features ”, Using this combination to achieve complex nonlinear scene description . Because of the simple structure of the model , The training of this approach / The estimated calculation cost is relatively small ; however , Feature selection is a labor-intensive task , And require relevant personnel to have a deeper understanding of the business .

Another idea of modeling is " Complex models + Simple features “. That is to weaken the importance of feature engineering , Using complex nonlinear models to learn the relationship between features , Enhance the ability of expression . The deep neural network model is such a nonlinear model .

The image above shows an image with an input layer , An output layer , Deep neural networks with two hidden layers . One of the models has 9 Nodes .

The introduction of neural network is very detailed in many literatures , Now take the above picture as an example , Let's focus on backpropagation The derivation process of the algorithm .

backpropagation Very similar to the gradient method , It's essentially finding the partial derivative of each parameter , Then find the next search point in the direction of the partial derivative , With ${W_{04}}$ For example ：

Combine the above derivation with , You can get ${W_{04}}$ The gradient direction of ：

The other iterative processes are similar to the gradient descent method .

It is worth noting that , although DNN The requirements for feature engineering are relatively low , But the training time is more complicated , The explainability of tangent weight is very poor , Not easy to debug. therefore , For a new application , A better way is to use Logistic Regression This kind of linear model began to be applied , When iteration matures , Try again DNN Model .

【 original 】 Deep neural network (Deep Neural Network, DNN) More articles about

1. use matlab Training deep neural networks for digital classification Training a Deep Neural Network for Digit Classification

   This example shows how to use Neural Network Toolbox to train a deep neural network to classify ima ...

2. How deep neural networks see you , On selfie What a Deep Neural Network thinks about your #selfie

   Convolutional Neural Networks are great: they recognize things, places and people in your personal p ...

3. Wu enda 《 Deep learning 》- The first course (Neural Networks and Deep Learning)- The fourth week ： Deep neural network (Deep Neural Networks)- Course notes

   The fourth week : Deep neural network (Deep Neural Networks) 4.1 Deep neural network (Deep L-layer neural network) There are some functions , Only very deep neural networks can learn , And the shallower model is ...

4. Neural Networks and Deep Learning（week4）Deep Neural Network - Application（ Image classification ）

   Deep Neural Network for Image Classification: Application Pre implemented code , Keep it locally dnn_app_utils_v3.py import n ...

5. Lesson one (Neural Networks and Deep Learning), The fourth week （Deep Neural Networks） —— 3.Programming Assignments: Deep Neural Network - Application

   Deep Neural Network - Application Congratulations! Welcome to the fourth programming exercise of the ...

6. Lesson one (Neural Networks and Deep Learning), The fourth week （Deep Neural Networks）——2.Programming Assignments: Building your Deep Neural Network: Step by Step

   Building your Deep Neural Network: Step by Step Welcome to your third programming exercise of the de ...

7. Artificial neural network Artificial Neural Network

   2017-12-18 23:42:33 One . What is deep learning Deep learning (deep neural network) It's a branch of machine learning , It is an attempt to use multiple processing layers consisting of complex structures or multiple non-linear transformations to process data in a high level ...

8. A Survey of Model Compression and Acceleration for Deep Neural Network when s

   A Survey of Model Compression and Acceleration for Deep Neural Network when s In this paper, the compression methods of deep neural network are summarized , It can be divided into parameter repair ...

9. Neural Networks and Deep Learning（week4）Building your Deep Neural Network: Step by Step

   Building your Deep Neural Network: Step by Step You will use the following functions to build a deep neural network for image classification . Use something like relu This is a nonlinear element to improve your model structure ...

Random recommendation

1. [C#] C# Knowledge review - characteristic Attribute

   C# Knowledge review - characteristic Attribute [ Blogger ] Anti bony boy [ Original address ]http://www.cnblogs.com/liqingwen/p/5911289.html Catalog Feature brief Use features characteristic ...

2. Poj1852

   The question is : The maximum value of all ants walking down from the stick in the shortest time ( When did the last ant come down ) Of all the ants , The situation that takes the longest time PS: There's no need to think about two ants meeting and turning back ( It can be directly thought that the two of them did not influence each other ) # ...

3. ASP.NET Razor——Razor brief introduction

   ASP.NET Razor - Mark Razor Not a programming language . It's a server-side markup language . What is? Razor? Razor It's a kind of tagging grammar , Allows you to translate server based code (Visual Basic and ...

4. mysql Full text search FULLTEXT

   FULLTEXT Indexes establish FULLTEXT Index Syntax establish table Time to create fullText Indexes CREATE TABLE table_name( column1 data_type, column2 ...

5. QT Source code analysis （ from QApplication Start ）

   QT Source code analysis Reprinted from :http://no001.blog.51cto.com/1142339/282130 today , When talking to classmates , When it comes to the Qt Source code problems , I found myself right Qt The understanding of mechanism is too little , and ...

6. Customize View Realize the Gobang game

   The road to success is not crowded at all , Because there are too few people who insist . --- A sentence from Jane's book I will take three days off in the future, and I will get married on November 11 , I have to say, it's a big hole , Last year, there were 10 God , This year it has shrunk to 3 God , We have to rush to the eleventh . I've been ...

7. About Vue Array operation

   Vue The implementation code of array operation is as follows : const aryMethods = ['push', 'pop', 'shift', 'unshift', 'splice', 'sort', 'reverse ...

8. 【2013Esri Wonderful cases of global user Conference 】GIS for Philadelphia’s Finest -- Philadelphia police GIS

     Industry sector : police   Have 6000 The Philadelphia police department, full of officers and detectives , It took three years , Built the Philadelphia police GIS, Now you can summarize what happens every day ( Pictured 1), And dynamic hot spot analysis ( Pictured 2). The regional commander can easily check the activities of police officers ...

9. PLSQL_ Data structure type analysis （ Concept ）

   2014-06-02 Created By BaoXinjian

10. What is Vertical Align?

    https://css-tricks.com/what-is-vertical-align/ ************************************************* CSS ...