The linear model is expressed by the current combination of features “ result - Feature set ” The correspondence between . Because of the limited expressive power of linear model , In practice , Only by adding “ Feature calculation ” To optimize the model . such as , In the advertisement CTR In application , except “ Title Length 、 Description length 、 Position 、 advertisement id,cookie“ And so on , And there's a lot of combinatorial features ( such as ” Position -cookie“ It indicates the user's preference for bits ). in fact , Now many search engine advertising systems use Logistic Regression Model ( linear ), And one of the most important jobs of the model team is “ Feature Engineering (feature engineering)”.
The idea of a linear model is “ Simple model + Complex features ”, Using this combination to achieve complex nonlinear scene description . Because of the simple structure of the model , The training of this approach / The estimated calculation cost is relatively small ; however , Feature selection is a labor-intensive task , And require relevant personnel to have a deeper understanding of the business .
Another idea of modeling is " Complex models + Simple features “. That is to weaken the importance of feature engineering , Using complex nonlinear models to learn the relationship between features , Enhance the ability of expression . The deep neural network model is such a nonlinear model .
The image above shows an image with an input layer , An output layer , Deep neural networks with two hidden layers . One of the models has 9 Nodes .
The introduction of neural network is very detailed in many literatures , Now take the above picture as an example , Let's focus on backpropagation The derivation process of the algorithm .
backpropagation Very similar to the gradient method , It's essentially finding the partial derivative of each parameter , Then find the next search point in the direction of the partial derivative , With ${W_{04}}$ For example :
Combine the above derivation with , You can get ${W_{04}}$ The gradient direction of :
The other iterative processes are similar to the gradient descent method .
It is worth noting that , although DNN The requirements for feature engineering are relatively low , But the training time is more complicated , The explainability of tangent weight is very poor , Not easy to debug. therefore , For a new application , A better way is to use Logistic Regression This kind of linear model began to be applied , When iteration matures , Try again DNN Model .
【 original 】 Deep neural network (Deep Neural Network, DNN) More articles about
- use matlab Training deep neural networks for digital classification Training a Deep Neural Network for Digit Classification
This example shows how to use Neural Network Toolbox to train a deep neural network to classify ima ...
- How deep neural networks see you , On selfie What a Deep Neural Network thinks about your #selfie
Convolutional Neural Networks are great: they recognize things, places and people in your personal p ...
- Wu enda 《 Deep learning 》- The first course (Neural Networks and Deep Learning)- The fourth week : Deep neural network (Deep Neural Networks)- Course notes
The fourth week : Deep neural network (Deep Neural Networks) 4.1 Deep neural network (Deep L-layer neural network) There are some functions , Only very deep neural networks can learn , And the shallower model is ...
- Neural Networks and Deep Learning(week4)Deep Neural Network - Application( Image classification )
Deep Neural Network for Image Classification: Application Pre implemented code , Keep it locally dnn_app_utils_v3.py import n ...
- Lesson one (Neural Networks and Deep Learning), The fourth week (Deep Neural Networks) —— 3.Programming Assignments: Deep Neural Network - Application
Deep Neural Network - Application Congratulations! Welcome to the fourth programming exercise of the ...
- Lesson one (Neural Networks and Deep Learning), The fourth week (Deep Neural Networks)——2.Programming Assignments: Building your Deep Neural Network: Step by Step
Building your Deep Neural Network: Step by Step Welcome to your third programming exercise of the de ...
- Artificial neural network Artificial Neural Network
2017-12-18 23:42:33 One . What is deep learning Deep learning (deep neural network) It's a branch of machine learning , It is an attempt to use multiple processing layers consisting of complex structures or multiple non-linear transformations to process data in a high level ...
- A Survey of Model Compression and Acceleration for Deep Neural Network when s
A Survey of Model Compression and Acceleration for Deep Neural Network when s In this paper, the compression methods of deep neural network are summarized , It can be divided into parameter repair ...
- Neural Networks and Deep Learning(week4)Building your Deep Neural Network: Step by Step
Building your Deep Neural Network: Step by Step You will use the following functions to build a deep neural network for image classification . Use something like relu This is a nonlinear element to improve your model structure ...
Random recommendation
- [C#] C# Knowledge review - characteristic Attribute
C# Knowledge review - characteristic Attribute [ Blogger ] Anti bony boy [ Original address ]http://www.cnblogs.com/liqingwen/p/5911289.html Catalog Feature brief Use features characteristic ...
- Poj1852
The question is : The maximum value of all ants walking down from the stick in the shortest time ( When did the last ant come down ) Of all the ants , The situation that takes the longest time PS: There's no need to think about two ants meeting and turning back ( It can be directly thought that the two of them did not influence each other ) # ...
- ASP.NET Razor——Razor brief introduction
ASP.NET Razor - Mark Razor Not a programming language . It's a server-side markup language . What is? Razor? Razor It's a kind of tagging grammar , Allows you to translate server based code (Visual Basic and ...
- mysql Full text search FULLTEXT
FULLTEXT Indexes establish FULLTEXT Index Syntax establish table Time to create fullText Indexes CREATE TABLE table_name( column1 data_type, column2 ...
- QT Source code analysis ( from QApplication Start )
QT Source code analysis Reprinted from :http://no001.blog.51cto.com/1142339/282130 today , When talking to classmates , When it comes to the Qt Source code problems , I found myself right Qt The understanding of mechanism is too little , and ...
- Customize View Realize the Gobang game
The road to success is not crowded at all , Because there are too few people who insist . --- A sentence from Jane's book I will take three days off in the future, and I will get married on November 11 , I have to say, it's a big hole , Last year, there were 10 God , This year it has shrunk to 3 God , We have to rush to the eleventh . I've been ...
- About Vue Array operation
Vue The implementation code of array operation is as follows : const aryMethods = ['push', 'pop', 'shift', 'unshift', 'splice', 'sort', 'reverse ...
- 【2013Esri Wonderful cases of global user Conference 】GIS for Philadelphia’s Finest -- Philadelphia police GIS
Industry sector : police Have 6000 The Philadelphia police department, full of officers and detectives , It took three years , Built the Philadelphia police GIS, Now you can summarize what happens every day ( Pictured 1), And dynamic hot spot analysis ( Pictured 2). The regional commander can easily check the activities of police officers ...
- PLSQL_ Data structure type analysis ( Concept )
2014-06-02 Created By BaoXinjian
- What is Vertical Align?
https://css-tricks.com/what-is-vertical-align/ ************************************************* CSS ...