Nodejs The biggest feature is based on event driven and asynchronous concurrent operation . Most people know nodejs It's a new platform for network background services , Can be very convenient to provide background services ; In addition to network development , Actually nodejs For offline file concurrent processing is also very convenient , differ C++、java, utilize nodejs Can quickly build read-write framework , Implementation of file processing operations .

This paper introduces a common file format in program development (csv) To deal with . stay nodejs In the development package provided by the official website , There's a lot about csv My bag , The most downloaded is CSV package , Here's how to use CSV Packages handle files .

Download the package first :

npm install csv

Use the examples on the official website :

//node samples/string.js
var csv =require('csv');
csv() .from('"1","2","3","4"\n"a","b","c","d"') .to(console.log )
//Output: //1,2,3,4
//a,b,c,d

The first thing to do is to quote csv modular ,require('csv'), After the references , The encapsulated methods and properties can be used directly .csv() Equivalent to instantiating an object ,.from and .to All are csv The way of encapsulation

.from() Method : seeing the name of a thing one thinks of its function , Is to read data from the source file , Parameters can be passed directly to strings as above , Also like the following advanced application, the path to the source file .

.to() Method : Yes, from form Method , It can be output to the console , It can also be output to the target file . This example is output to the console .

 // node samples/sample.js
var fs =require('fs');
var csv =require('csv');
csv()
.from.stream(fs.createReadStream(__dirname+'/sample.in'))
.to.path(__dirname+'/sample.out')
.transform(function(row){
row.unshift(row.pop());
return row;
})
.on('record',function(row,index){
console.log('#'+index+' '+JSON.stringify(row));
})
.on('close',function(count){
// when writing to a file, use the 'close' event
// the 'end' event may fire before the file has been written
console.log('Number of lines: '+count);
})
.on('error',function(error){
console.log(error.message);
});
// Output:
// #0 ["2000-01-01","20322051544","1979.0","8.8017226E7","ABC","45"]
// #1 ["2050-11-27","28392898392","1974.0","8.8392926E7","DEF","23"]
// Number of lines: 2

.transform() Method : Handle from Read out the data , Write to the target file after processing ; Parameters can be passed to callback functions , In the callback function , You can write methods for processing data .

.on() :  Event monitoring mechanism ,

'record' Listen for each line of record read out , Parameters in row It's a row of data ,index It's the number of data , Be careful , The number of data is just the number of data read out , Not the number of data processed , because nodejs Is asynchronous operation , So processing data is batch , The program will send the data batch by batch to transform, Wait until a batch is processed before writing to the target file , So the number read out index Often greater than the number of files that have been written , How to capture the number of written files after processing , It will be dealt with later .

'end'  Event triggered after processing a file ;

'error'  While processing data , An event triggered by an exception occurs ;

'close' Events triggered when a file is closed ;

Advanced usage :

csv The advanced part of the module is that it reads data by line , The data read out is in the form of objects , The column name is the name of the object key, The value is value, So it's very easy to take the value of each column , Don't want to c++ You have to parse fields as well .

.from.path(strSrcPath, {header: true, columns: true})

from The second parameter setting in the method separates the columns ;

When writing the processed data to a new file , You can also set parameters .

.to.path(strDestPath,{

header: true

,lineBreaks:'windows'

,newColumns: true

,end: false

,columns:fields

,flags : 'w'

})

Here is a description of each parameter

*   `delimiter`   csv The separator for each column of the file ;

*   `columns`     Target file the columns of the target file ,

*   `header`      With or without a header ;

*    `lineBreaks`   Separate the identification between the lines , Yes  'auto', 'unix', 'mac', 'windows', 'unicode' several ;

*   `flags`        Create a new target file or append data to an existing target file ,’w’ newly build ;’a’ Additional ;

*   `end`         In response to end Before the news , The document is not writable ;

csv Breakpoint continuation

this csv The only drawback of the module is that it doesn't support breakpoints , That is, a file is not processed completely , Next time we'll deal with it . The key to solve this problem is to capture the number of data that the program has processed , Mentioned earlier 'record' You can only listen to the number that the program reads , It does not represent the number of files that have been written . Followed csv The source code of the module , Found that as long as you modify the source code , Listen to the event of each batch write to the target file , The problem is solved .

this.emit("writen", this.state.countWriten);

stay csv.js Add custom listening Events ,countWriten It is the number of target files written each time . And then call csv Add this event to the module .

.on('writen', function(index){

console.log("rtn", index);

})

In this way, the number of written files can be captured in real time , This data can be written to a cursor file and recorded . With a little modification ,csv The module supports breakpoints , It's perfect to use .

nodejs:csv Module analysis of more related articles

  1. Python in csv Module analysis

    The import module import csv 2. Read csv file file1 = open('test1.csv', 'rb') reader = csv.reader(file1) rows = [row for ...

  2. nodejs Modular system ( The example analysis exprots and module.exprots)

    Preface : A good workman does his work well , You must sharpen your tools first . The modular system is nodejs The sharp tool of organizing and managing code is also the way to call third-party code , This article will explain in detail nodejs Modular system . At the end of the article, an example is given exprots and module.exprots ...

  3. Nodejs cluster Module in-depth exploration

    from the outside to the inside HTTP The server is used to respond to requests from clients , When the number of client requests increases gradually, there are many kinds of server processing mechanisms , Such as tomcat The multithreading .nginx And so on . And for node for , Because it also uses event loops and asynchrony I/O machine ...

  4. TypeScript and Node Module parsing strategy

    In general, when we code in modules , Always import other modules , Usually we use the following syntax : import { A } from './a'; // ES6 grammar import { A } from 'a'; var A = re ...

  5. NodeJS Web modular

    NodeJS Web modular In this paper, nodeJS Of http Basic usage of module , Implement simple server and client classic Web framework Client: The client usually refers to the browser , adopt HTTP Protocol sends request to server (request) S ...

  6. nodejs cheerio Module extraction html The page content

    nodejs cheerio Module extraction html The page content 1. nodejs cheerio Module extraction html The page content 1.1. Find the target element 1.2. Beautify text output 1.3. Extract answer text 1.4. Final ...

  7. Python And CSV modular

    1. CSV brief introduction CSV(Comma Separated Values) Is the comma separator text format , Commonly used in Excel And database import and export ,Python Standard library CSV Module provides read and write CSV The object of the format file . ...

  8. Python And CSV file (CSV modular )

    Python And CSV file (CSV modular )   1.CSV file CSV( Comma separated values ) Formats are the most common import and export formats for spreadsheets and databases . No, “CSV standard ”, So the format is operationally defined by many read-write applications . Lack of standards means ...

  9. python Standard library :csv modular

    Original address :http://www.bugingcode.com/blog/python_csv.html csv Modules are used to read CSV Format ( Separate values with commas ) Data files for ,CSV The file format is often in Microsoft's Exc ...

Random recommendation

  1. Tomcat Container virtual path settings

    1.[ Official documents ] stay tomcat\conf Next server.xml Find <Host name="localhost" appBase="webapps" u ...

  2. solve IIS The problem of slow access for the first time after process recycling

    IIS There is a mechanism , The default will be IIS After a certain period of free time , Recycle the application pool , This time period is IIS6 Default is 20 minute , stay IIS7 Default is 1740 minute . Neither configuration is reasonable , When the application pool is recycled , First visit ...

  3. [ Tools ]json conversion

    Abstract I saw an introduction in the garden this week JsonCSharpClassGenerator Articles on this tool , I think it's very practical , In the current project json The most used , So when converting the corresponding class , It's quite frequent , So I studied this work ...

  4. Open product development (OPD):Archi Chinese tools download

    Updated version of this article has been moved to  http://www.zhoujingen.cn/blog/3378.html ------------------------------ stay OPD in , We used ArchiMa ...

  5. iOS Development Use RMStore Simplify internal purchase code + Internal purchase order verification

    Now a lot of app All of them add in app purchase , There are many tutorials about the generation and setting of Apple certificate on the Internet , I won't go into details here , Recommend a few personal feel that the more detailed website : http://www.jianshu.com/p/86ac7d ...

  6. uWSGI third :uWSGI collocation Nginx Use

    http://www.nowamagic.net/academy/detail/1330334 In the last introduction uWSGI To deploy Django Program , But in a production environment, it's just uWSGI It's not enough. ,N ...

  7. Python Script controlled WebDriver Common operations &lt; 8、 ... and &gt; Simple object positioning

    Contents of this part , Will be in WebDriver in , Demonstration of positioning element method , Yes, it will Selenium in Selenese Element positioning command WebDriver A combination of the methods used in Selenium In the element localization method review may refer to : & ...

  8. Yii 2.0 install

    adopt Composer install Be careful : php The best version is 5.5 above ! 1. download  Yii2 Advanced application templates for  , Then unzip it to a Web Accessible folder . 2. download Composer-Setup.exe , ...

  9. ZendStudio-12.5.0-win32.win32.x86_64.msi Official version and cracking tools

    I tried a lot of tools on the Internet , Finally, the tool downloaded succeeded , Previous N All the tools failed Try it yourself , Indicates valid !!! ZendStudio-12.5.0-win32.win32.x86_64.msi Official version download address :   hundred ...

  10. utilize shell Script implementation nginx Of logs Log segmentation

    Nginx It's a very light weight Web The server , Small volume . High performance . Speed and many other advantages . But there are also shortcomings , For example, the access log file generated by it has always been a , It doesn't cut automatically , If there's a lot of traffic , take Result in log file capacity ...