1.  Let me write it out front

I/O ,I yes Input ( Input ) Abbreviation ,O yes Output ( Output ) Abbreviation , as everyone knows , People want to communicate with each other , You need to speak a language that you can understand each other , For example, we all speak English together .

If humans want to communicate with computers , It also needs a common language , And computers only know binary 0101 Code , However, human beings find it difficult to understand and learn this language , So there is a “ Translation machine “—— Advanced programming language , such as C#,Java etc. , High level programming language can translate high-level language code interpretation into binary code through an interpreter program , So the computer can understand our commands , Do what we want it to do .

Again , There are also ways of communication between programs , That's it I/O.

I/O In a sense, it's an application and a file , network connections , Console A set of communication API.

stay Java.io The most important thing in the bag is 5 A class and an interface .5 A class refers to File、OutputStream、InputStream、Writer、Reader; An interface refers to Serializable. If you master these, you will master them Java I/O The essence of .

As mentioned above , If we want to manipulate files , have access to File Related to the class , To create a new file , Copy , Move , Traverse , Delete and other operations .

If you want to operate on the contents of the file , We can consider two ways , One way is to read a byte stream , One way is to read characters .

You may wonder when and which way is better ?

My suggestion :

If there is no Chinese in the operation file content , Generally, byte stream is OK (InputStream and OutputStream), Of course, with character stream (Reader and Writer) No problem .

But if there is Chinese in the operation file , So we have to use character stream to operate , Pay attention to character encoding when using it .

Why is that ?

Because a Chinese character occupies at least 2 Bytes , An English text usually occupies at least one byte .

Java in I/O Operation mainly refers to the use of Java Input , Output operation . Java be-all I/O All mechanisms are based on data flow for input and output , These data streams represent flowing sequences of character or byte data .

  •   Byte stream : The smallest data unit in a data stream is a byte  
  •   Character stream : The smallest data unit in a data stream is a character , Java The characters in are Unicode code , One character takes two bytes .

To display Chinese characters correctly , You need at least a combination of two bytes to read and write , If you use byte stream to read and write , What you may see is random code .

When different numbers of bytes are combined, there is no way Character encoding . such as GB2312 Perfect support for simplified Chinese and English , A Chinese character occupies 2 Bytes , One English letter also takes up 2 Bytes .

Discussion details about character coding , I happened to write an interesting blog when I was a freshman , When you execute this command ,C The language printed out is “ I love you! “” Three words

printf("%c%c%c%c%c%c\n",,,,,,);

Display print results :

If you are interested, you can have a look at C Two in the language ASCLL Codes can represent Chinese characters

JDK6  Online Chinese documents :http://tool.oschina.net/apidocs/apidoc?api=jdk-zh

JDK7: http://tool.oschina.net/apidocs/apidoc?api=jdk_7u4

2. Java Of I/O The basic classification of

I/O Problems are unavoidable in any programming language , so to speak I/O The problem is the core of human-computer interaction , because I/O It is the main channel for machines to acquire and exchange information .

We write program code through high-level language I/O operation , So as to realize human-computer interaction .

stay Java.io The most important thing in the bag is 5 A class and an interface .

5 A class refers to File、OutputStream、InputStream、Writer、Reader; An interface refers to Serializable.

If you master these, you will master them Java I/O The essence of .

Java Of I/O Operation class in package java.io Next , It's about 80 Classes , But these classes can be roughly divided into four groups , Namely :

  1. Byte based I/O Interface :InputStream and OutputStream
  2. Character based I/O Interface :Writer and Reader
  3. Disk based I/O Interface :File
  4. Based on network operation I/O Interface :Socket

The first two groups are mainly based on the data format of the transmitted data , The latter two groups are mainly based on the mode of data transmission , although Socket Class is not in java.io It's a bag , But I still put them together , Because I personally think I/O The core issue is either data format impact I/O operation , Or the transmission mode I/O operation , That is to say, what kind of data should be written to where ,I/O It's just a way for people to interact with machines or between machines , In addition to being able to do this interaction , What we focus on is how to improve the efficiency of its operation , The data format and transmission mode are the most critical factors affecting the efficiency . Our later analysis is also based on these two factors .———— Excerpt from IBM  Learning documents

java.io There is... In the bag 4 It's a basic category :InputStream、OutputStream And Reader、Writer class , They deal with byte streams and character streams, respectively .

All kinds of other streams are created by this 4 It's a derivative of .

stay JDK 1.4 And then introduced NIO,NIO Compared with traditional IO The flow has improved , One of the highlights is the introduction of the buffer concept .

2.1 File class

About File What we need to pay attention to is , It can be used to represent not only a file but also a folder .

File Class can write a utility class .

Now if we need to query all the files in a certain folder, it is .java A list of types of files

So we need two parameters , One is to query the folder path name , The second parameter is the type of file

If it's not recursive , It can be called like this

List<File> fileList=SmartFileUtils.lookFollder("C:\\Users\\fairy\\Pictures\\ National Geographic Museum ",".jpg");// Only query image files Not a recursive 
//List<File> fileList=SmartFileUtils.lookFollder("C:\\Users\\fairy\\Pictures\\ National Geographic Museum ");// Query all folders and files Not a recursive 
for (File file : fileList) { System.out.println(file.getAbsolutePath()); }

Recursively query the current folder, including all the java File list , We can call this :

public static void main(String[] args) {
FileTreeInfo fileTreeInfo=SmartFileUtils.watchFolder(".",".java");// Query only java file recursive 
 //FileTreeInfo fileTreeInfo=SmartFileUtils.watchFolder(".");// Query all folders and files recursive 
 System.out.println(fileTreeInfo.toString());
}

If you want to get the folder and file collection list of query results separately , In this way, you can call

List<File> fileList=fileTreeInfo.fileList;
List<File> folderList=fileTreeInfo.folderList;

SmartFileUtils.java

import java.io.File;
import java.io.FilenameFilter;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Iterator;
import java.util.List;
import java.util.regex.Pattern; /**
* @author fairy Query the files in the folder
***/
public class SmartFileUtils {
/**
* View a list of all files and folders in a folder
* No recursion
* */
public final static List<File> lookFollder(String startDir) {
return lookFollderInfo(new File(startDir), ".*");
}
/**
* View a list of files and folders of a specified type under a folder
* No recursion
* */
public final static List<File> lookFollder(String startDir, String regex) {
return lookFollderInfo(new File(startDir), ".*\\" + regex);
}
private final static List<File> lookFollderInfo(File rootFolder, final String regex) {
return Arrays.asList(
rootFolder.listFiles(new FilenameFilter() {
private Pattern pattern = Pattern.compile(regex);
@Override
public boolean accept(File dir, String name) {
// TODO Auto-generated method stub
return pattern.matcher(name).matches();
}
}));
} /**
* View a list of all the files and folders in a folder
* Recursive traversal
* */
public final static FileTreeInfo watchFolder(String startDir) {
return watchDirs(new File(startDir), ".*");
}
/**
* View a list of files and folders of a specified type under a folder
* Recursive traversal
* */
public final static FileTreeInfo watchFolder(String startDir, String regex) {
return watchDirs(new File(startDir), ".*" + regex);
}
private final static FileTreeInfo watchDirs(File startDir, String regex) {
FileTreeInfo resultInfo = new FileTreeInfo(); for (File item : startDir.listFiles()) {
if (item.isDirectory()) {
resultInfo.folderList.add(item);
resultInfo.addAll(watchDirs(item, regex));
} else {
if (item.getName().matches(regex)) {
resultInfo.fileList.add(item);
}
}
}
return resultInfo;
}
/**
* TreeInfo
***/
public static class FileTreeInfo implements Iterable<File> {
public List<File> fileList = new ArrayList<File>();
public List<File> folderList = new ArrayList<File>(); public Iterator<File> iterator() {
return fileList.iterator();
} void addAll(FileTreeInfo other) {
fileList.addAll(other.fileList);
folderList.addAll(other.folderList);
} public String toString() {
return "dirs:" + PPrint.pFormat(folderList) + "\n\nfiles:" + PPrint.pFormat(fileList);
}
}
}

In the above code, we refer to a formatted print collection tool class :

import java.util.Collection;
public class PPrint {
public static String pFormat(Collection<?> c) {
if(c.size()==0) return "[]";
StringBuilder result=new StringBuilder();
for (Object elem:c) {
if(c.size()!=1) {
result.append("\n ");
}
result.append(elem);
}
if(c.size()!=1) {
result.append("\n");
}
result.append("]");
return result.toString();
}
}

Conversion between bytes and characters

In addition, data persistence or network transmission is in bytes , So there must be character to byte or byte to character conversion .

Byte to character conversion is required , The conversion process of reading is shown in the figure below :

Byte stream <---- read -----> Character stream

InputStreamReader Class is a bridge from byte to character ,StreamDecoder It is the implementation class that completes the decoding of byte to character

InputStream To Reader To specify the coded character set , Otherwise, the operating system default character set will be used , It's very likely that there will be confusion .

Writing is a similar process, as shown in the figure below :

chart :  Byte stream <---- Write -----> Character stream

adopt OutputStreamWriter Class completion , Character to byte encoding , from StreamEncoder Complete coding process .

5. Java IO  The basic method of flow

5.1 InputStream  Basic approach

5.2 OutputStream  Basic approach

5.3 Reader Basic approach

5.4 Writer  Basic approach

6. Java  Use of common streams

6.1 Node flow type

FileInputStreamTest.java

package com.xingyun.io;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.net.URISyntaxException; public class FileInputStreamTest { public final static String getCurrentFilePath =new File("").getAbsolutePath()+"\\src\\com\\xingyun\\io\\"+FileInputStreamTest.class.getSimpleName()+".java";
//getCurrentFilePath: C:\Users\fairy\Documents\EclipseWorkspace\JavaSEBaseSample\src\com\xingyun\io\InputStreamTest.java public static void main(String[] args) throws IOException, URISyntaxException {
// TODO Auto-generated method
int b=0;//read byte count
FileInputStream fileInputStream=null;
try {
fileInputStream=new FileInputStream(getCurrentFilePath);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
System.out.println("File not found");
System.exit(-1);
}
try {
long num=0;
while((b=fileInputStream.read())!=-1) {
System.out.print((char)b);
num++;
}
System.out.println("--read over------");
System.out.println("read "+num+" byte");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally {
fileInputStream.close();
}
}
}

FileOutputStreamTest.java

package com.xingyun.io;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException; public class FileOutputStreamTest { public final static String getCurrentFilePath =new File("").getAbsolutePath()+"\\src\\com\\xingyun\\io\\"+FileOutputStreamTest.class.getSimpleName()+".java";
public final static String writeNewFilePath=new File("").getAbsolutePath()+"\\src\\com\\xingyun\\io\\"+FileOutputStreamTest.class.getSimpleName()+"New.txt"; public static void main(String[] args) {
// TODO Auto-generated method stub
int b=0;
FileInputStream fileInputStream=null;
FileOutputStream fileOutputStream=null; try {
fileInputStream=new FileInputStream(getCurrentFilePath);
fileOutputStream=new FileOutputStream(writeNewFilePath);
while ((b=fileInputStream.read())!=-1) {
fileOutputStream.write(b);
} } catch (FileNotFoundException e) {
// TODO Auto-generated catch block
System.out.println("can't find the file");
} catch (IOException e) {
// TODO Auto-generated catch block
System.out.println("read or write error");
}finally {
try {
System.out.println("read and write success");
fileInputStream.close();
fileOutputStream.close();
} catch (IOException e) {
// TODO Auto-generated catch block
System.out.println("close error");
System.exit(-1);
}
} } }

FileReaderTest.java

package com.xingyun.io;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException; public class FileReaderTest { private final static String getCurrentFilePath =new File("").getAbsolutePath()+"\\src\\com\\xingyun\\io\\"+FileReaderTest.class.getSimpleName()+".java"; public static void main(String[] args) {
// TODO Auto-generated method stub
FileReader fileReader=null;
int c=0;
try {
fileReader=new FileReader(getCurrentFilePath);
while((c=fileReader.read())!=-1) {
System.out.print((char)c);
}
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally {
try {
fileReader.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}

FileWriterTest.java

package com.xingyun.io;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException; public class FileWriterTest { public final static String getCurrentFilePath =new File("").getAbsolutePath()+"\\src\\com\\xingyun\\io\\"+FileWriterTest.class.getSimpleName()+".java";
public final static String writeNewFilePath=new File("").getAbsolutePath()+"\\src\\com\\xingyun\\io\\"+FileWriterTest.class.getSimpleName()+"New.txt";
public static void main(String[] args) {
// TODO Auto-generated method stub
int b=0;
FileReader fileReader=null;
FileWriter fileWriter=null; try {
fileReader=new FileReader(getCurrentFilePath);
fileWriter=new FileWriter(writeNewFilePath);
while ((b=fileReader.read())!=-1) {
fileWriter.write(b);
}
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
System.out.println("can't find the file");
} catch (IOException e) {
// TODO Auto-generated catch block
System.out.println("read or write error");
}finally {
try {
System.out.println("read and write success");
fileReader.close();
fileWriter.close();
} catch (IOException e) {
// TODO Auto-generated catch block
System.out.println("close error");
System.exit(-1);
}
} } }

6.2  Processing stream type

6.2.1  Buffer flow

BufferedInputStreamTest.java

package com.xingyun.io;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException; public class BufferedInputStreamTest { public final static String getCurrentFilePath =new File("").getAbsolutePath()+"\\src\\com\\xingyun\\io\\"+BufferedInputStreamTest.class.getSimpleName()+".java"; public static void main(String[] args) {
// TODO Auto-generated method stub
FileInputStream fileInputStream=null;
BufferedInputStream bufferedInputStream=null;
try {
fileInputStream=new FileInputStream(getCurrentFilePath);
bufferedInputStream=new BufferedInputStream(fileInputStream,100);
int c=0;
System.out.println(bufferedInputStream.read());
System.out.println(bufferedInputStream.read());
bufferedInputStream.mark(100);//jump to 100 location
for (int i = 0; i <10&&((c=bufferedInputStream.read())!=-1); i++) {
System.out.print((char)c+" ");
}
System.out.println("");
System.out.println("------------------");
bufferedInputStream.reset();//return to 100 location
for (int i = 0; i <10&&((c=bufferedInputStream.read())!=-1); i++) {
System.out.print((char)c+" ");
}
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally {
try { fileInputStream.close();
bufferedInputStream.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
} }

BufferedWriterAndBufferedReaderTest.java

package com.xingyun.io;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException; public class BufferedWriterAndBufferedReaderTest { public final static String getCurrentFilePath =new File("").getAbsolutePath()+"\\src\\com\\xingyun\\io\\"+BufferedWriterAndBufferedReaderTest.class.getSimpleName()+"New.txt"; public static void main(String[] args) {
// TODO Auto-generated method stub
BufferedWriter bufferedWriter=null;
BufferedReader bufferedReader=null;
try {
bufferedWriter=new BufferedWriter(new FileWriter(getCurrentFilePath),100);
bufferedReader=new BufferedReader(new FileReader(getCurrentFilePath),100);
String string=null;
for(int i=0;i<100;i++) {
string=String.valueOf(Math.random());
bufferedWriter.write(string);
bufferedWriter.newLine();
}
bufferedWriter.flush();
while ((string=bufferedReader.readLine())!=null) {
System.out.println(string);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally {
try {
bufferedWriter.close();
bufferedReader.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
} } }

6.2.2 Converted flow

OutputStreamWriterTest.java

package com.xingyun.io;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader; public class InputStreamReaderTest { public static void main(String[] args) {
// TODO Auto-generated method stub
InputStreamReader inputStreamReader = new InputStreamReader(System.in); BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
String string = null;
try {
string = bufferedReader.readLine();
while (string != null) {
if (string.equals("exit"))
break;
else {
System.out.println(string.toUpperCase());
string = bufferedReader.readLine();
} }
System.out.println(bufferedReader.readLine());
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
try {
inputStreamReader.close();
bufferedReader.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
} }

InputStreamReaderTest.java

package com.xingyun.io;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader; public class InputStreamReaderTest { public static void main(String[] args) {
// TODO Auto-generated method stub
InputStreamReader inputStreamReader = new InputStreamReader(System.in);
BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
String string = null;
try {
string = bufferedReader.readLine();
while (string != null) {
if (string.equals("exit"))
break;
else {
System.out.println(string.toUpperCase());
string = bufferedReader.readLine();
} }
System.out.println(bufferedReader.readLine());
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
try {
inputStreamReader.close();
bufferedReader.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
} }

6.2.3 Print stream

PrintStreamTest.java

package com.xingyun.io;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.PrintStream; public class PrintStreamTest { public final static String getCurrentFilePath = new File("").getAbsolutePath() + "\\src\\com\\xingyun\\io\\"
+ PrintStreamTest.class.getSimpleName() + "New.txt"; public static void main(String[] args) {
// TODO Auto-generated method stub
PrintStream printStream = null;
FileOutputStream fileOutputStream=null;
try {
fileOutputStream = new FileOutputStream(getCurrentFilePath);
printStream = new PrintStream(fileOutputStream);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} if (printStream != null) {
System.setOut(printStream);
}
//it will write into a file but not console screen
int line = 0;
for (char c = 0; c <= 60000; c++) {
System.out.print(c + "");
if (line++ >= 100) {
System.out.println();
line = 0;
}
}
//it will write into a file but not console screen
System.out.println("write into file success");
try {
fileOutputStream.close();
printStream.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

PrintStreamTest2.java

package com.xingyun.io;
import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.io.PrintStream; public class PrintStreamTest2 { public static void main(String[] args) {
// TODO Auto-generated method stub
String filename = args[0];
//you must type the file path as args
//C:\Users\fairy\Documents\EclipseWorkspace\JavaSEBaseSample\src\com\xingyun\io\PrintStreamTest2.java
System.out.println("please type file path which one file you need to print");
if (filename != null) {
list(filename, System.out);
}
} public static void list(String f, PrintStream printStream) {
BufferedReader bufferedReader = null;
try {
bufferedReader = new BufferedReader(new FileReader(f));
String string = null;
while ((string = bufferedReader.readLine()) != null) {
printStream.println(string);
}
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
try {
bufferedReader.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
} }

PrintStreamTest3.java

package com.xingyun.io;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.util.Date; public class PrintStreamTest3 { public final static String getCurrentFilePath =new File("").getAbsolutePath()+"\\src\\com\\xingyun\\io\\"+PrintStreamTest3.class.getSimpleName()+"New.log"; public static void main(String[] args) {
// TODO Auto-generated method stub
String string=null;
System.out.println("Please type message:");
BufferedReader bufferedReader=new BufferedReader(new InputStreamReader(System.in),100);
FileWriter fileWriter=null;
PrintWriter printWriter=null; try {
fileWriter=new FileWriter(getCurrentFilePath,true);
printWriter=new PrintWriter(fileWriter); while((string=bufferedReader.readLine())!=null) {
if(string.equals("exit"))break;
System.out.println(string.toUpperCase());
printWriter.println("-------------------");
printWriter.println(string.toUpperCase());
printWriter.flush();
}
printWriter.println("-------------"+new Date()+"--------");
printWriter.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally {
printWriter.close();
}
}
}

6.2.4  Object  flow

Tips:

transient  This keyword decorated variable is not processed during serialization

serializable  Make your own entity class support serialization and deserialization

externalizable  Interface , Handle serialization and deserialization on your own

ObjectOutputStreamTest.java

package com.xingyun.io;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable; public class ObjectOutputStreamTest { public final static String getCurrentFilePath =new File("").getAbsolutePath()+"\\src\\com\\xingyun\\io\\"+ObjectOutputStreamTest.class.getSimpleName()+"New.log"; public static void main(String[] args) throws IOException, ClassNotFoundException {
// TODO Auto-generated method stub
TestObject testObject=new TestObject();
testObject.k=8; //write
FileOutputStream fileOutputStream=new FileOutputStream(getCurrentFilePath);
ObjectOutputStream objectOutputStream=new ObjectOutputStream(fileOutputStream);
objectOutputStream.writeObject(testObject);
objectOutputStream.flush();
objectOutputStream.close();
fileOutputStream.close(); //read
FileInputStream fileInputStream=new FileInputStream(getCurrentFilePath);
ObjectInputStream objectInputStream=new ObjectInputStream(fileInputStream);
TestObject tReaderObject= (TestObject)objectInputStream.readObject();
System.out.println("------"+tReaderObject.i+"----------"+tReaderObject.j+"------"+tReaderObject.d+"---------"+tReaderObject.k);
objectInputStream.close();
fileInputStream.close(); } static class TestObject implements Serializable{
/**
*
*/
private static final long serialVersionUID = -5736467218763256621L;
int i=10;
int j=9;
double d=2.30;
// int k=18;
transient int k=18;//don't deal with the value
}
}

7. IO tuning

Here's the disk I/O And the Internet I/O Some of the common optimization techniques are summarized as follows :

disk I/O Optimize

Performance test

Our applications usually need to access the disk to read data , And disk I/O It's usually time consuming , We need to judge I/O Is it a bottleneck , We have some parameters for reference :

For example, we can stress test the application to see the performance of the system I/O wait Whether the index is normal , For example, the test machine has 4 individual CPU, So ideal I/O wait Parameters should not exceed 25%, If exceeded 25% Words ,I/O It's likely to be a performance bottleneck for applications .Linux Under the operating system, you can use iostat Command view .

Usually we're judging I/O Another parameter that we will also look at in performance is IOPS, Our application needs the lowest IOPS How much is the , And our disks are IOPS Can we meet our requirements . Of each disk IOPS Usually in a range , This is also related to the size and access mode of the data block stored on the disk . But it's mainly determined by the speed of the disk , The higher the speed of the disk, the higher the speed of the disk IOPS And the higher .

Now in order to improve the disk I/O Performance of , It's usually called RAID Technology , Is to combine different disks to improve I/O performance , There are many kinds of RAID technology , Each of these RAID Technical right I/O The performance improvement will be different , You can use a RAID Factor to represent , The read and write throughput of the disk can be achieved through iostat Command to get , So we can work out a theoretical IOPS value , The formula is as follows, so :

( Number of disks * Of each disk IOPS)/( Disk read throughput +RAID factor * Disk write throughput )=IOPS

Please refer to resources for details of this formula  Understanding Disk I/O.

promote I/O performance

Upgrade disk I/O The common ways to improve performance are :

  1. Add cache , Reduce disk access
  2. Optimize disk management system , Design the optimal disk access policy , And disk addressing strategy , This is at the bottom operating system level .
  3. Design reasonable disk storage data block , And the strategy for accessing these data blocks , This is at the application level . For example, we can design indexes for stored data , Speed up and reduce disk access by addressing index , In addition, asynchronous and non blocking methods can be used to speed up disk access efficiency .
  4. Apply reasonable RAID Policy promotion disk IO, Each of these RAID We can use the following table to show the difference :
surface 2.RAID Strategy
disk array explain
RAID 0 Data is written to multiple disk arrays on average , Write data and read data are parallel , So the disk is IOPS It can double .
RAID 1 RAID 1 The main function of is to improve the security of data , It copies a single piece of data into multiple disk arrays . It doesn't promote IOPS But there are multiple backups of the same data . It is usually used in the situation of high data security .
RAID 5 This design is a compromise between the first two , It writes the data to the disk with the total number of all disk arrays minus one on average , Write the parity information of this data to another disk . If one of the disks is damaged , This data can be recovered from the data of other disks and the parity information of this data .
RAID 0+1 Like the name , It is to group the data according to the backup situation , One copy of data is written to multiple backup disk groups at the same time , At the same time, multiple groups will read and write in parallel .

The Internet I/O Optimize

The Internet I/O Optimization usually has some basic processing principles :

  1. One is to reduce the number of network interactions : To reduce the number of network interactions, we usually set a cache at both ends of the network interaction , such as Oracle Of JDBC The driver , Provides a query of SQL Caching of results , On both the client side and the database side , Can effectively reduce the access to the database . About Oracle JDBC Memory management can refer to 《 Oracle JDBC memory management 》. In addition to setting the cache, there's another way , Merge access requests : For example, when querying the database , We need to check 10 individual id, I can check one at a time id, You can also check it once 10 individual id. For example, when you visit a page, there will be multiple js or css The file of , We can have multiple js Files merged in one HTTP In the link , Each file is separated by a comma , And then send it to the back end Web Server according to this URL link , And then split the files , Then package it and send it back to the front-end browser . These are commonly used to reduce the network I/O The way to .
  2. Reduce the amount of data transmitted on the network : The way to reduce the amount of network data is usually to compress the data and then transmit it , Such as HTTP In request , Usually Web The server will request Web page gzip After compression, it is transmitted to the browser . And by designing simple protocols , Try to get useful value information by reading protocol header . For example, when designing agent programs , Yes 4 Layer agents and 7 Layer agents are used to avoid reading the entire communication data to obtain the required information as far as possible .
  3. Minimize coding : Usually on the Internet I/O Data transmission is in the form of bytes , That is to say, serialization is usually required . But the data we send for transmission is in the form of characters , From character to byte must be encoded . But this coding process is time-consuming , So we have to go through the Internet I/O When transmitting , Try to send it directly in bytes . That is, try to convert characters to bytes in advance , Or reduce the conversion from character to byte .
  4. Design appropriate interaction mode according to the application scenario : The so-called interaction scenarios mainly include synchronous and asynchronous blocking and non blocking modes , More on this below .

Knowledge expansion

New input / Output (NIO) Ku is in JDK 1.4 Introduced in .NIO Make up for the original I/O Deficiency , It's in the standard Java High speed 、 Block oriented I/O. By defining classes that contain data , And by processing the data in blocks ,NIO You can take advantage of low-level optimizations without using native code , This is the original I/O What a package can't do .

Reference article

Java I/O summary    http://www.importnew.com/23708.html

In depth analysis Java I/O How it works  https://www.ibm.com/developerworks/cn/java/j-lo-javaio/

NIO introduction  https://www.ibm.com/developerworks/cn/education/java/j-nio/j-nio.html

Java NIO And IO  contrast : http://ifeve.com/java-nio-vs-io/

Java IO More related articles in the stream summary

  1. Java Medium IO flow - Introduction

    Preface Hello, everyone , I'm tangyuan , What I bring to you today is <Java Medium IO flow - Introduction >, Hopefully that helped , thank you because Java Of IO There are many classes , And that led to when I first started learning , It's a mess , Every time I use it, I search the Internet , result ...

  2. Java Basics 17:Java IO Stream summary

    More content, please pay attention to WeChat official account. [Java Technical Jianghu ] This is an Ali Java Engineer's technical station , Author Huang Xiaoxie , focus Java Related technology :SSM.SpringBoot.MySQL. Distributed . middleware . colony .Linux ...

  3. Java IO Flow learning summarizes four : Buffer flow -BufferedReader、BufferedWriter

    In the last article Java IO Flow learning summarizes three : Buffer flow -BufferedInputStream.BufferedOutputStream Introduces the byte stream in buffer stream , This article focuses on the word stream in buffer stream Buffered ...

  4. 《 Two 》Java IO Introduction to the classification of flow

    One . According to the flow direction, it can be divided into input flow and output flow : Note that the input and output streams are relative to the program . Output : Put the program ( Memory ) Output the contents of to disk . CD and other storage devices         Input : Read external data ( disk . Data from storage devices such as optical disks ) To the program ...

  5. Java IO Flow and explain in detail ( One )—— Brief introduction

    Files are transmitted as streams in programs . So use Java To transfer files, you have to use Java IO flow . 1. The concept and function of flow flow : Represents any data source object that has the ability to produce data or the receiver object that has the ability to receive data <Thinkin ...

  6. Java:IO Stream and file base

    Java:IO Stream and file base explain : The content of this chapter will be continuously updated , You can pay attention to it and give me some advice , Thank you very much! . Into the stream What is flow flow : An ordered sequence of bytes from source to destination . stay Java in , An object from which a sequence of bytes can be read ...

  7. java IO Flow and explain in detail

    The concept and function of flow Study Java IO, It has to be mentioned that JavaIO flow . Flow is a set of sequential , Set of bytes with start and end points , Is the general term or abstraction of data transmission . That is, the transmission of data between two devices is called streaming , The essence of stream is data transmission , According to the data transmission ...

  8. Java IO Stream learning summary

    Java Class or interface related to stream operations : Java Flow class diagram structure : The concept and function of flow Flow is a set of sequential , Set of bytes with start and end points , Is the general term or abstraction of data transmission . That is, the transmission of data between two devices is called streaming , The essence of stream is data transmission , According to the data transmission ...

  9. uncover Java IO Streaming flush() The veil of mystery

    We are using Java IO Streaming OutputStream.PrintWriter -- when , It's going to be used a lot flush() Method . Just like caching in network hardware , Streams can also be cached in software , That is, directly in Java Cache in code . ...

Random recommendation

  1. docker4dotnet #2 Containerized hosts

    .NET Since the ape met the little whale , I feel that my skill has been greatly improved . Part 1 <docker4dotnet #1 All men are mortal & Hello world > This article introduces how to use the Windows The above configuration Docker for Window ...

  2. Javascript Constructor and new command 2

    A typical object-oriented programming language ( such as C++ and Java), There is " class "(class) The concept . So-called " class " It's the template of the object , Object is " class " Example . however , stay J ...

  3. 93. Restore IP Addresses

    subject : Given a string containing only digits, restore it by returning all possible valid IP address com ...

  4. Liunx Next Tomcat+MYSQL+Nginx To configure

    Environmental Science :centos6.4 min # Install the compiler library and dependent modules  yum -y install gcc gcc-c++ autoconf automake make yum -y install zlib z ...

  5. openstack Development step-by-step

    Set up your Open Stack There are several ways to deploy openstack, Devstack is easily for developer ...

  6. MTK MOTA Upgrade steps

    MOTA It has its own server,MTK I've finished in , Sure MTK application , And then move to its own server Up . 1. open ProjectConfig.mk middle MTK_SYSTEM_UPDATE_SUP ...

  7. STM32 - GPIO

    Bought one. STM32F4 Development board , I want to learn the above things again Quick pass : One .GPIO control void GPIO_DeInit(GPIO_TypeDef* GPIOx); // Put a certain IO Port back to default /* ...

  8. HDU 3594 Cactus ( Strong connectivity + Cactus picture )

    < Topic link > < Reproduced in >>> > The main idea of the topic : Here's a picture for you , Let you judge if he is a cactus . The condition of cactus graph is : 1. It's a strongly connected graph . 2. Each edge in the cactus graph belongs to only one ...

  9. Codeforces Round #497 (Div. 2)

    Codeforces Round #497 (Div. 2) https://codeforces.com/contest/1008 A #include<bits/stdc++.h> u ...

  10. ( turn )Linux One of the kernel parameters arp_ignore and arp_announce

    original text :https://blog.csdn.net/ccy19910925/article/details/79960599 One .arp_ignore and arp_announce Introduce arp_ignore ...