About KNN, I'm glad to see This article , Write very well , I'm not going to go over it here . Just stick the code , There are small changes .（ Turned out to be python2 Version of , Here instead python3 Of , Mainly print）

Environmental Science ：win7 32bit + spyder + anaconda3.5

One 、 The first step

```# -*- coding: utf-8 -*-
"""
Created on Sun Nov 6 16:09:00 2016
"""
#Input:
# newInput: Data points to be measured (1xM)
# dataSet: Known data (NxM)
# labels: Tags of known data (1xM)
# k: The number of nearest data points selected
#
#Output:
# Classification label of data points to be tested
#
from numpy import *
# creat a dataset which contain 4 samples with 2 class
def createDataSet():
# creat a matrix: each row as a sample
group = array([[1.0, 0.9], [1.0, 1.0], [0.1, 0.2], [0.0, 0.1]])
labels = ['A', 'A', 'B', 'B']
return group, labels
#classify using KNN
def KNNClassify(newInput, dataSet, labels, k):
numSamples = dataSet.shape[0] # row number
# step1:calculate Euclidean distance
# tile(A, reps):Constract an array by repeating A reps times
diff = tile(newInput, (numSamples, 1)) - dataSet
squreDiff = diff**2
squreDist = sum(squreDiff, axis=1) # sum if performed by row
distance = squreDist ** 0.5
#step2:sort the distance
# argsort() returns the indices that would sort an array in a ascending order
sortedDistIndices = argsort(distance)
classCount = {}
for i in range(k):
# choose the min k distance
voteLabel = labels[sortedDistIndices[i]]
#step4:count the times labels occur
# when the key voteLabel is not in dictionary classCount,
# get() will return 0
classCount[voteLabel] = classCount.get(voteLabel, 0) + 1
#step5:the max vote class will return
maxCount = 0
for k, v in classCount.items():
if v > maxCount:
maxCount = v
maxIndex = k
return maxIndex
# test
dataSet, labels = createDataSet()
testX = array([1.2, 1.0])
k = 3
outputLabel = KNNClassify(testX, dataSet, labels, 3)
print("Your input is:", testX, "and classified to class: ", outputLabel)
testX = array([0.1, 0.3])
k = 3
outputLabel = KNNClassify(testX, dataSet, labels, 3)
print("Your input is:", testX, "and classified to class: ", outputLabel)
```

Running results ：

The handwriting recognition database information used in here download . About the introduction of information in the above blog has also been introduced very clearly .

```# -*- coding: utf-8 -*-
"""
Created on Sun Nov 6 16:09:00 2016
"""
#Input:
# newInput: Data points to be measured (1xM)
# dataSet: Known data (NxM)
# labels: Tags of known data (1xM)
# k: The number of nearest data points selected
#
#Output:
# Classification label of data points to be tested
#
from numpy import *
#classify using KNN
def KNNClassify(newInput, dataSet, labels, k):
numSamples = dataSet.shape[0] # row number
# step1:calculate Euclidean distance
# tile(A, reps):Constract an array by repeating A reps times
diff = tile(newInput, (numSamples, 1)) - dataSet
squreDiff = diff**2
squreDist = sum(squreDiff, axis=1) # sum if performed by row
distance = squreDist ** 0.5
#step2:sort the distance
# argsort() returns the indices that would sort an array in a ascending order
sortedDistIndices = argsort(distance)
classCount = {}
for i in range(k):
# choose the min k distance
voteLabel = labels[sortedDistIndices[i]]
#step4:count the times labels occur
# when the key voteLabel is not in dictionary classCount,
# get() will return 0
classCount[voteLabel] = classCount.get(voteLabel, 0) + 1
#step5:the max vote class will return
maxCount = 0
for k, v in classCount.items():
if v > maxCount:
maxCount = v
maxIndex = k
return maxIndex
# convert image to vector
def img2vector(filename):
rows = 32
cols = 32
imgVector = zeros((1, rows * cols))
fileIn = open(filename)
for row in range(rows):
for col in range(cols):
imgVector[0, row * 32 + col] = int(lineStr[col])
return imgVector
## step 1: Getting training set
print("---Getting training set...")
dataSetDir = 'F:\\Techonolgoy\\ Algorithm learning \\KNN\\ Advanced \\'
trainingFileList = os.listdir(dataSetDir + 'trainingDigits') # load the training set
numSamples = len(trainingFileList)
train_x = zeros((numSamples, 1024))
train_y = []
for i in range(numSamples):
filename = trainingFileList[i]
# get train_x
train_x[i, :] = img2vector(dataSetDir + 'trainingDigits/%s' % filename)
# get label from file name such as "1_18.txt"
label = int(filename.split('_')[0]) # return 1
train_y.append(label)
## step 2: Getting testing set
print("---Getting testing set...")
testingFileList = os.listdir(dataSetDir + 'testDigits') # load the testing set
numSamples = len(testingFileList)
test_x = zeros((numSamples, 1024))
test_y = []
for i in range(numSamples):
filename = testingFileList[i]
# get train_x
test_x[i, :] = img2vector(dataSetDir + 'testDigits/%s' % filename)
# get label from file name such as "1_18.txt"
label = int(filename.split('_')[0]) # return 1
test_y.append(label)
return train_x, train_y, test_x, test_y
# test hand writing class
def testHandWritingClass():
train_x, train_y, test_x, test_y = loadDataSet()
## step 2: training...
print("step 2: training...")
pass
## step 3: testing
print("step 3: testing...")
numTestSamples = test_x.shape[0]
matchCount = 0
for i in range(numTestSamples):
predict = KNNClassify(test_x[i], train_x, train_y, 3)
if predict == test_y[i]:
matchCount += 1
accuracy = float(matchCount) / numTestSamples
## step 4: show the result
print("step 4: show the result...")
print('The classify accuracy is: %.2f%%' % (accuracy * 100))
testHandWritingClass()
```

Running results ：

## About KNN Of python3 More articles on Implementation

1. Python3 Realize the classical algorithm of machine learning （ One ）KNN

One .KNN summary K-( most ) Nearest neighbor algorithm KNN(k-Nearest Neighbor) It is one of the simplest methods in data mining classification technology . It has high precision . The advantage of being insensitive to outliers , It is suitable for dealing with discrete numerical data , But it has very ...

2. Python3 k- Proximity algorithm （KNN）

# -*- coding: utf-8 -*- """ Created on Fri Dec 29 13:13:44 2017 @author: markli " ...

3. Machine learning practice python3 K a near neighbor （KNN） Algorithm implementation

I've seen all the techniques and footstones of Taiwan University , But no programming has been , Now I plan to combine Zhou Zhihua's < machine learning >, Let's go through the real battle of machine learning , The original book is python2 Of , But I feel python3 Better to use , So I plan to use python ...

4. Python3 Realize the classical algorithm of machine learning （ Two ）KNN Implement a simple OCR

One . Preface 1.ocr summary OCR (Optical Character Recognition, Optical character recognition ) Electronic equipment ( For example, a scanner or a digital camera ) Check the characters printed on the paper , By detecting dark . The bright pattern determines its shape , however ...

5. kNN.py Source code and notes （python3.x）

import numpy as npimport operatorfrom os import listdirdef CerateDataSet():        group = np.array( ...

6. KNN Recognize the number on the image and python Realization

The leader asked me to input it manually every day BI Data in the system and judge whether the data is abnormal , If there's something unusual , Then detect whether it is a system problem or a business problem . In order to free hands , I decided to write a program to complete the task of entering and managing cockpit data every day . First of all, we recorded a set of scripts with the key wizard to put the ...

7. Machine learning practice notes (Python Realization )-01-K Nearest neighbor algorithm (KNN)

--------------------------------------------------------------------------------------- This series of articles is < machine ...

8. The first 2 Chapter KNN Algorithm notes _ function classify0

< Machine learning practice > Knowledge point notes Directory K- Nearest neighbor algorithm (KNN) thought : 1, Calculate the distance between the unknown sample and all known samples 2, Sort by increasing distance , Before the election K Samples (K<20) 3, in the light of K A sample of statistics of the emergence of each category ...

9. machine learning --kNN Algorithms recognize handwritten letters

This paper mainly uses kNN Algorithm for feature extraction of letter image , classification . The contents are as follows : kNN Algorithm and related Python Module introduction Feature extraction of letter image kNN Algorithm implementation kNN Algorithm analysis One .kNN Algorithm is introduced K a near neighbor (kNN,k ...

## Random recommendation

1. C# DataTable In detail

Reprint someone else's reprint , The original author doesn't know It's often used in projects DataTable, If DataTable Use it properly , It can not only make the program simple and practical , And it can improve performance , Achieve twice the result with half the effort , Right now DataTable The use of skills for a ...

2. Netizens forward very complete LISTCTL Control to use

author :lixiaosan Time :04/06/2006 The following is not stated ,listctrl Default view Style is report Related classes and processing functions MFC:CListCtrl class SDK: With "ListV ...

3. 01 knapsack URAL 1073 Square Country

Subject portal /* The question : ask n At least it can be the sum of the squares of a few numbers 01 knapsack :j*j The question of whether or not to buy our land Explain in detail :http://www.cnblogs.com/vongang/archive/2011/10/07/2 ...

4. Source code maintenance basic command diff_patch

Source code maintenance basic command diff------ Generate source code patches diff [ Command line options ] Source file A new file -r Recursively process the corresponding directory -N Include new files to patch-u Output unified format (unified format), This kind of case ...

5. Android Explanation of mobile phone instruction operation

Someone asked me a question about Android mobile phone root The question of whether or not , She said clearly iTools Shows that root jurisdiction , But she just can't run it on the phone root The powers of the App Such as titanium backup and so on . I told her that the best way to confirm is to adb Instructions ...

6. WCF Request data : Exceeded incoming message (65536) Maximum message size quota for . To increase the quota , Use... On the corresponding binding element MaxReceivedMessageSize attribute .

Usually , Write well WCF After service, they will use their own WCFClient Tools to test , In the process of data request , If the amount of returned data exceeds the default received value, an abnormal error will be prompted as shown in the figure . The error is also very clear , It's just the question of modifying the size of the received value ...

7. Import theano Failure .“cannot import name gof”

Follow the online tutorial to install theano after ,import theano There is an error ."cannot import name gof". I found a lot of tutorials on the Internet , It doesn't work . Finally, find the following tutorial . Personal test ...

8. 201521123117 《Java Programming 》 The first 4 Weekly learning summary

1. This week's learning summary Q1.1 Try to use mind map to summarize knowledge about inheritance . Q1.2  Summarize other lessons in the usual way . answer : Control the visibility of 4 Access modifiers private -- Only visible to this class public -- Right ...

9. C3P0 and DBCP The difference between

C3P0 and DBCP The difference between C3P0 It's an open source JDBC Connection pool , It implements data sources and JNDI binding , Support JDBC3 Normative and JDBC2 Standard extension of . At present, the open source projects using it are Hibernate,Spring etc. .     ...

10. python2.7 Next use logging Module log to the terminal display garbled problem solving

I just flipped 2 I used it years ago python2.7 Write a crawler program , The main function is to put the intellectual property offices of various cities / Patent related projects on the website of the patent office . Project notice , Regularly crawl and analyze , Assist enterprises in project application . This is not about the implementation of the crawler function , It's today ...