mongoDB Of Journaling Log function and common log Logs are different ,mongoDB Also have log journal , It simply records the startup information of the database on the server 、 Slow query record 、 Database exception information 、 The client connects to the database server 、 Disconnection and other information .Journaling The log function is mongoDB It's a very important function , It ensures that the database server is powered down unexpectedly 、 Data integrity in case of natural disasters, etc . Even though mongoDB Other backup measures such as replica set are also provided ( We will analyze later ), but Journaling The function of is indispensable in the production environment , It relies on smaller CPU And memory consumption , What it brings is the persistence and stability of the database . This chapter will analyze Journaling The functional details involved .Journaling workflow ?

Several important storage views

Journaling The function uses two important memory views :private view and shared view. Both memory views are created through MMAP( Memory mapping ) To achieve , Among them the private view The memory modification of the map will not affect the disk ,shared view Changes in the data on the disk can affect the files on the disk , The system will refresh periodically shared view The data in the file is transferred to disk .

(1) shared view stay mongoDB In the startup process , The operating system maps the data files on the disk to... In memory shared view, The operating system just does the mapping , Data is not immediately loaded into memory ,mongoDB Will load data to shared view.

(2)private view The memory view is where data is saved for read operations , yes mongoDB The first place to save new writes .

(3) On disk Journaling Log files , It's where write operations are persisted .mongoDB This file is read when the instance starts .

The process of handling write operations

When a write operation occurs ,mongoDB First write the data to... In memory private view It's about , Then batch copy the write operations to journal,journal Write operations are stored on a file on disk , Make it persistent ,journal Each entry in the log file describes which bytes on the data file are changed by the write operation .

When the above steps are completed ,mongoDB And then we'll use journal Log to update shared view Data in , At this time, the data in memory and the data on disk become inconsistent . By default 60 second ,mongoDB Periodically, the operating system will shared view The data changed in is refreshed to disk , Keep the data on disk consistent with the data in memory .

After refreshing the changed data in memory to disk ,mongoDB Will be deleted journal All write operations after this point in time .

Last ,mongoDB Will shared view And private view To resynchronize , consistency .

mongoDB Of journaling Log function , stay 2.0 After version, it starts by default , It can be found in the example mongod Startup time , Control through the startup option ; In the steps mentioned above , One place is to batch write operations periodically to journal Log file , The size of this cycle is determined by the optional startup parameter journalCommitInterval Controlled , The default value is 100ms.

mongoDB after 60s Refresh the changed data in memory to disk , This value is set by starting the optional parameter syncdelay Controlled . These default values generally apply in most cases , Don't change it easily . Through the above analysis , The database server still has 100ms The risk of losing data , because journaling The write cycle of the log to disk is 100ms, If just a batch of write operations are still in memory , I didn't get to journaling On the corresponding file on the disk , The server broke down suddenly , These write operations in memory are lost .

mongoDB When it starts , Special initialization of a thread, continuous loop , Used to come from within a certain period of time defer Queue to get persistent data and write it to disk journal( journal ) and mongofile( data ) It's about , Of course, it's not written to disk when the user adds a record , So press mongodb The developer said , It doesn't cause performance degradation , Because I saw the code and found , When doing CUD In operation , Record (Record type ) All put into defer In the queue for delay batch (group commit) Commit to write .

All in all mongoDB Using memory mapping technology to complete these functions , Need reference unix Memory mapping in environment programming MMAP, file IO Etc. programming knowledge .

mongoDB Research Notes :journaling More articles on ensuring data integrity in case of unexpected failures

  1. journaling Ensure data integrity in case of unexpected failure

    mongoDB Research Notes :journaling Ensure data integrity in case of unexpected failure   mongoDB Of Journaling Log function and common log Logs are different ,mongoDB Also have log journal , It simply records the data ...

  2. mongoDB Research Notes : Replication set failover mechanism

    The data synchronization described above (http://www.cnblogs.com/guoyuanwei/p/3293668.html) It is equivalent to the backup strategy in traditional database ,mongoDB On this basis, there is the function of automatic fail over . In reply ...

  3. MongoDB Practical Study ( Four ):MongoDB Of Journaling Log function

    mongoDB Of Journaling Log function and common log Logs are different ,mongoDB Also have log journal , It simply records the startup information of the database on the server . Slow query record . Database exception information . The client connects to the database server ...

  4. ORACLE Summary of database recovery under different failures

    ORACLE Summary of database recovery under different failures 1. Files lost or damaged in non Archive Mode --1.1 Data files -- Start the state of the database to MOUNT-- Recovery method : Through a full backup of the previously created database , Fix the entire database , But after the backup ...

  5. mongoDB Research Notes : Overview of replica sets

    Self learning , For reference only : The database will always encounter all kinds of failure scenarios , If the network connection is disconnected . Power off, etc , Even though journaling The log function also provides the function of data recovery , but journaling Usually for a single node , Only single node data can be guaranteed ...

  6. mongoDB Research Notes : Fragment cluster deployment

    In the analysis of the previous articles, the replica set solves the problem of database backup and automatic fail over , But there are two aspects of the business around the database that are becoming more and more important . One is how to store massive data ? The second is how to read and write massive data efficiently ? Although replica sets can also perform read-write analysis , Such as ...

  7. mongoDB Research Notes : Replication set data synchronization mechanism

    http://www.cnblogs.com/guoyuanwei/p/3279572.html  Overview of replica sets , In general, we have a concept of replica set , But one of the most important functions of replica set is how to realize data synchronization ? With this question ...

  8. mongodb 2.6.6 stay window10 Installation below

    First of all, thank you for knowing , Second, sigh WIN10 My fork , Compatibility is not bad . First , download mongodb 2.6.6 Installation package , unfortunately , The official website is too slow , You have to register . But also good , I'm a good man :http://pan.baidu.com/ ...

  9. MongoDB A series of :CentOS7.2 Lower installation mongoDB3.2.8

    I've been tossing about again recently MongoDB, Post the installation configuration related commands  1. download  wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70- ...

Random recommendation

  1.  Make a complete record of Oracle 11.2.0.4 Single instance printing PSU The process of patching

    In this paper, it is recorded that PSU The whole process , The purpose of this paper is to experience the database PSU The whole process of patching . 1.OPatch Replace with the latest version 2. Database software application 19121551 Patches 3. Database application patch 4. verification PSU Whether the patch is applied successfully 1.OPa ...

  2. css Authoritative guide study notes

    2016-08-03 1, Inherit In general, most box model properties cannot be inherited . Such as :padding .margin .border .background Can't inherit . Inheritance value , There is no particularity at all .( It's the lowest priority ) ...

  3. RabbitMQ No link exception

    Link code Exception in project startup report Local main Method link report exception Find out the reason online Problem description and solution : There are many reasons online , The ultimate reason is that we can't connect to the database . 1. View firewall 2.tomcat Whether the port is shielded 3. Check out the company ...

  4. [c++] Program memory partition understanding

    Global and static data areas : Used to store global variables and static variables ( Global and local variables ) Constant data area : Used to store constant data Code section : For storing code Stack : For local variables and function parameters Pile up : Programmers apply for ( The part of programmer control ,new/delet ...

  5. Shell A special variable in the

    $ Represents the current process , When using echo $$ It will output the current shell Of pid echo $$ List of special variables Variable meaning $0 File name of the current script $n Parameters passed to a script or function .n It's a number , Indicates the number of parameters . example ...

  6. mysql--user surface

    mysql database Use cases :mysql  mysql> show tables; +---------------------------+ | Tables_in_mysql | +------- ...

  7. H5 id Selectors and class Selectors

    11-id Selectors and class Selectors The first paragraph The second paragraph is The third paragraph --> The first paragraph The second paragraph is The third paragraph <!DOCTYPE html> <html lang=&qu ...

  8. Xamarin + MvvmCross Simple case Part 2

    MvvmCross Speaking of MvvmCross, First of all Mvvm,Mvvm yes Mvc A deformation of the frame . The corresponding ones are Model.View and ViewModel layer . The relationship between the three layers is like this : Model The data layer is the data layer , real ...

  9. An Introduction to OAuth 2

    PostedJuly 21, 2014 1.1mviews SECURITY API CONCEPTUAL Mitchell Anicas Introduction OAuth 2 is an aut ...

  10. Winform Key capture

    Reference resources :http://blog.csdn.net/zhensoft163/article/details/4239796 Download link Method 1: Using the form of  KeyDown event private void F ...