python机器学习理论与实战(六)支持向量机


Posted in Python onJanuary 19, 2018

上节基本完成了SVM的理论推倒,寻找最大化间隔的目标最终转换成求解拉格朗日乘子变量alpha的求解问题,求出了alpha即可求解出SVM的权重W,有了权重也就有了最大间隔距离,但是其实上节我们有个假设:就是训练集是线性可分的,这样求出的alpha在[0,infinite]。但是如果数据不是线性可分的呢?此时我们就要允许部分的样本可以越过分类器,这样优化的目标函数就可以不变,只要引入松弛变量python机器学习理论与实战(六)支持向量机即可,它表示错分类样本点的代价,分类正确时它等于0,当分类错误时python机器学习理论与实战(六)支持向量机,其中Tn表示样本的真实标签-1或者1,回顾上节中,我们把支持向量到分类器的距离固定为1,因此两类的支持向量间的距离肯定大于1的,当分类错误时python机器学习理论与实战(六)支持向量机肯定也大于1,如(图五)所示(这里公式和图标序号都接上一节)。

python机器学习理论与实战(六)支持向量机

(图五)

       这样有了错分类的代价,我们把上节(公式四)的目标函数上添加上这一项错分类代价,得到如(公式八)的形式:

python机器学习理论与实战(六)支持向量机

(公式八)

重复上节的拉格朗日乘子法步骤,得到(公式九):

python机器学习理论与实战(六)支持向量机

(公式九)

         多了一个Un乘子,当然我们的工作就是继续求解此目标函数,继续重复上节的步骤,求导得到(公式十):

 python机器学习理论与实战(六)支持向量机

(公式十)

         又因为alpha大于0,而且Un大于0,所以0<alpha<C,为了解释的清晰一些,我们把(公式九)的KKT条件也发出来(上节中的第三类优化问题),注意Un是大于等于0

python机器学习理论与实战(六)支持向量机 

      推导到现在,优化函数的形式基本没变,只是多了一项错分类的价值,但是多了一个条件,0<alpha<C,C是一个常数,它的作用就是在允许有错误分类的情况下,控制最大化间距,它太大了会导致过拟合,太小了会导致欠拟合。接下来的步骤貌似大家都应该知道了,多了一个C常量的限制条件,然后继续用SMO算法优化求解二次规划,但是我想继续把核函数也一次说了,如果样本线性不可分,引入核函数后,把样本映射到高维空间就可以线性可分,如(图六)所示的线性不可分的样本:

python机器学习理论与实战(六)支持向量机

(图六)

         在(图六)中,现有的样本是很明显线性不可分,但是加入我们利用现有的样本X之间作些不同的运算,如(图六)右边所示的样子,而让f作为新的样本(或者说新的特征)是不是更好些?现在把X已经投射到高维度上去了,但是f我们不知道,此时核函数就该上场了,以高斯核函数为例,在(图七)中选几个样本点作为基准点,来利用核函数计算f,如(图七)所示:

python机器学习理论与实战(六)支持向量机

(图七)

       这样就有了f,而核函数此时相当于对样本的X和基准点一个度量,做权重衰减,形成依赖于x的新的特征f,把f放在上面说的SVM中继续求解alpha,然后得出权重就行了,原理很简单吧,为了显得有点学术味道,把核函数也做个样子加入目标函数中去吧,如(公式十一)所示:

 python机器学习理论与实战(六)支持向量机

(公式十一) 

        其中K(Xn,Xm)是核函数,和上面目标函数比没有多大的变化,用SMO优化求解就行了,代码如下:

def smoPK(dataMatIn, classLabels, C, toler, maxIter): #full Platt SMO 
 oS = optStruct(mat(dataMatIn),mat(classLabels).transpose(),C,toler) 
 iter = 0 
 entireSet = True; alphaPairsChanged = 0 
 while (iter < maxIter) and ((alphaPairsChanged > 0) or (entireSet)): 
  alphaPairsChanged = 0 
  if entireSet: #go over all 
   for i in range(oS.m):   
    alphaPairsChanged += innerL(i,oS) 
    print "fullSet, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  else:#go over non-bound (railed) alphas 
   nonBoundIs = nonzero((oS.alphas.A > 0) * (oS.alphas.A < C))[0] 
   for i in nonBoundIs: 
    alphaPairsChanged += innerL(i,oS) 
    print "non-bound, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  if entireSet: entireSet = False #toggle entire set loop 
  elif (alphaPairsChanged == 0): entireSet = True 
  print "iteration number: %d" % iter 
 return oS.b,oS.alphas

下面演示一个小例子,手写识别。

      (1)收集数据:提供文本文件

      (2)准备数据:基于二值图像构造向量

      (3)分析数据:对图像向量进行目测

      (4)训练算法:采用两种不同的核函数,并对径向基函数采用不同的设置来运行SMO算法。

       (5)测试算法:编写一个函数来测试不同的核函数,并计算错误率

       (6)使用算法:一个图像识别的完整应用还需要一些图像处理的只是,此demo略。

完整代码如下:

from numpy import * 
from time import sleep 
 
def loadDataSet(fileName): 
 dataMat = []; labelMat = [] 
 fr = open(fileName) 
 for line in fr.readlines(): 
  lineArr = line.strip().split('\t') 
  dataMat.append([float(lineArr[0]), float(lineArr[1])]) 
  labelMat.append(float(lineArr[2])) 
 return dataMat,labelMat 
 
def selectJrand(i,m): 
 j=i #we want to select any J not equal to i 
 while (j==i): 
  j = int(random.uniform(0,m)) 
 return j 
 
def clipAlpha(aj,H,L): 
 if aj > H: 
  aj = H 
 if L > aj: 
  aj = L 
 return aj 
 
def smoSimple(dataMatIn, classLabels, C, toler, maxIter): 
 dataMatrix = mat(dataMatIn); labelMat = mat(classLabels).transpose() 
 b = 0; m,n = shape(dataMatrix) 
 alphas = mat(zeros((m,1))) 
 iter = 0 
 while (iter < maxIter): 
  alphaPairsChanged = 0 
  for i in range(m): 
   fXi = float(multiply(alphas,labelMat).T*(dataMatrix*dataMatrix[i,:].T)) + b 
   Ei = fXi - float(labelMat[i])#if checks if an example violates KKT conditions 
   if ((labelMat[i]*Ei < -toler) and (alphas[i] < C)) or ((labelMat[i]*Ei > toler) and (alphas[i] > 0)): 
    j = selectJrand(i,m) 
    fXj = float(multiply(alphas,labelMat).T*(dataMatrix*dataMatrix[j,:].T)) + b 
    Ej = fXj - float(labelMat[j]) 
    alphaIold = alphas[i].copy(); alphaJold = alphas[j].copy(); 
    if (labelMat[i] != labelMat[j]): 
     L = max(0, alphas[j] - alphas[i]) 
     H = min(C, C + alphas[j] - alphas[i]) 
    else: 
     L = max(0, alphas[j] + alphas[i] - C) 
     H = min(C, alphas[j] + alphas[i]) 
    if L==H: print "L==H"; continue 
    eta = 2.0 * dataMatrix[i,:]*dataMatrix[j,:].T - dataMatrix[i,:]*dataMatrix[i,:].T - dataMatrix[j,:]*dataMatrix[j,:].T 
    if eta >= 0: print "eta>=0"; continue 
    alphas[j] -= labelMat[j]*(Ei - Ej)/eta 
    alphas[j] = clipAlpha(alphas[j],H,L) 
    if (abs(alphas[j] - alphaJold) < 0.00001): print "j not moving enough"; continue 
    alphas[i] += labelMat[j]*labelMat[i]*(alphaJold - alphas[j])#update i by the same amount as j 
                  #the update is in the oppostie direction 
    b1 = b - Ei- labelMat[i]*(alphas[i]-alphaIold)*dataMatrix[i,:]*dataMatrix[i,:].T - labelMat[j]*(alphas[j]-alphaJold)*dataMatrix[i,:]*dataMatrix[j,:].T 
    b2 = b - Ej- labelMat[i]*(alphas[i]-alphaIold)*dataMatrix[i,:]*dataMatrix[j,:].T - labelMat[j]*(alphas[j]-alphaJold)*dataMatrix[j,:]*dataMatrix[j,:].T 
    if (0 < alphas[i]) and (C > alphas[i]): b = b1 
    elif (0 < alphas[j]) and (C > alphas[j]): b = b2 
    else: b = (b1 + b2)/2.0 
    alphaPairsChanged += 1 
    print "iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
  if (alphaPairsChanged == 0): iter += 1 
  else: iter = 0 
  print "iteration number: %d" % iter 
 return b,alphas 
 
def kernelTrans(X, A, kTup): #calc the kernel or transform data to a higher dimensional space 
 m,n = shape(X) 
 K = mat(zeros((m,1))) 
 if kTup[0]=='lin': K = X * A.T #linear kernel 
 elif kTup[0]=='rbf': 
  for j in range(m): 
   deltaRow = X[j,:] - A 
   K[j] = deltaRow*deltaRow.T 
  K = exp(K/(-1*kTup[1]**2)) #divide in NumPy is element-wise not matrix like Matlab 
 else: raise NameError('Houston We Have a Problem -- \ 
 That Kernel is not recognized') 
 return K 
 
class optStruct: 
 def __init__(self,dataMatIn, classLabels, C, toler, kTup): # Initialize the structure with the parameters 
  self.X = dataMatIn 
  self.labelMat = classLabels 
  self.C = C 
  self.tol = toler 
  self.m = shape(dataMatIn)[0] 
  self.alphas = mat(zeros((self.m,1))) 
  self.b = 0 
  self.eCache = mat(zeros((self.m,2))) #first column is valid flag 
  self.K = mat(zeros((self.m,self.m))) 
  for i in range(self.m): 
   self.K[:,i] = kernelTrans(self.X, self.X[i,:], kTup) 
   
def calcEk(oS, k): 
 fXk = float(multiply(oS.alphas,oS.labelMat).T*oS.K[:,k] + oS.b) 
 Ek = fXk - float(oS.labelMat[k]) 
 return Ek 
   
def selectJ(i, oS, Ei):   #this is the second choice -heurstic, and calcs Ej 
 maxK = -1; maxDeltaE = 0; Ej = 0 
 oS.eCache[i] = [1,Ei] #set valid #choose the alpha that gives the maximum delta E 
 validEcacheList = nonzero(oS.eCache[:,0].A)[0] 
 if (len(validEcacheList)) > 1: 
  for k in validEcacheList: #loop through valid Ecache values and find the one that maximizes delta E 
   if k == i: continue #don't calc for i, waste of time 
   Ek = calcEk(oS, k) 
   deltaE = abs(Ei - Ek) 
   if (deltaE > maxDeltaE): 
    maxK = k; maxDeltaE = deltaE; Ej = Ek 
  return maxK, Ej 
 else: #in this case (first time around) we don't have any valid eCache values 
  j = selectJrand(i, oS.m) 
  Ej = calcEk(oS, j) 
 return j, Ej 
 
def updateEk(oS, k):#after any alpha has changed update the new value in the cache 
 Ek = calcEk(oS, k) 
 oS.eCache[k] = [1,Ek] 
   
def innerL(i, oS): 
 Ei = calcEk(oS, i) 
 if ((oS.labelMat[i]*Ei < -oS.tol) and (oS.alphas[i] < oS.C)) or ((oS.labelMat[i]*Ei > oS.tol) and (oS.alphas[i] > 0)): 
  j,Ej = selectJ(i, oS, Ei) #this has been changed from selectJrand 
  alphaIold = oS.alphas[i].copy(); alphaJold = oS.alphas[j].copy(); 
  if (oS.labelMat[i] != oS.labelMat[j]): 
   L = max(0, oS.alphas[j] - oS.alphas[i]) 
   H = min(oS.C, oS.C + oS.alphas[j] - oS.alphas[i]) 
  else: 
   L = max(0, oS.alphas[j] + oS.alphas[i] - oS.C) 
   H = min(oS.C, oS.alphas[j] + oS.alphas[i]) 
  if L==H: print "L==H"; return 0 
  eta = 2.0 * oS.K[i,j] - oS.K[i,i] - oS.K[j,j] #changed for kernel 
  if eta >= 0: print "eta>=0"; return 0 
  oS.alphas[j] -= oS.labelMat[j]*(Ei - Ej)/eta 
  oS.alphas[j] = clipAlpha(oS.alphas[j],H,L) 
  updateEk(oS, j) #added this for the Ecache 
  if (abs(oS.alphas[j] - alphaJold) < 0.00001): print "j not moving enough"; return 0 
  oS.alphas[i] += oS.labelMat[j]*oS.labelMat[i]*(alphaJold - oS.alphas[j])#update i by the same amount as j 
  updateEk(oS, i) #added this for the Ecache     #the update is in the oppostie direction 
  b1 = oS.b - Ei- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.K[i,i] - oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.K[i,j] 
  b2 = oS.b - Ej- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.K[i,j]- oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.K[j,j] 
  if (0 < oS.alphas[i]) and (oS.C > oS.alphas[i]): oS.b = b1 
  elif (0 < oS.alphas[j]) and (oS.C > oS.alphas[j]): oS.b = b2 
  else: oS.b = (b1 + b2)/2.0 
  return 1 
 else: return 0 
 
def smoP(dataMatIn, classLabels, C, toler, maxIter,kTup=('lin', 0)): #full Platt SMO 
 oS = optStruct(mat(dataMatIn),mat(classLabels).transpose(),C,toler, kTup) 
 iter = 0 
 entireSet = True; alphaPairsChanged = 0 
 while (iter < maxIter) and ((alphaPairsChanged > 0) or (entireSet)): 
  alphaPairsChanged = 0 
  if entireSet: #go over all 
   for i in range(oS.m):   
    alphaPairsChanged += innerL(i,oS) 
    print "fullSet, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  else:#go over non-bound (railed) alphas 
   nonBoundIs = nonzero((oS.alphas.A > 0) * (oS.alphas.A < C))[0] 
   for i in nonBoundIs: 
    alphaPairsChanged += innerL(i,oS) 
    print "non-bound, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  if entireSet: entireSet = False #toggle entire set loop 
  elif (alphaPairsChanged == 0): entireSet = True 
  print "iteration number: %d" % iter 
 return oS.b,oS.alphas 
 
def calcWs(alphas,dataArr,classLabels): 
 X = mat(dataArr); labelMat = mat(classLabels).transpose() 
 m,n = shape(X) 
 w = zeros((n,1)) 
 for i in range(m): 
  w += multiply(alphas[i]*labelMat[i],X[i,:].T) 
 return w 
 
def testRbf(k1=1.3): 
 dataArr,labelArr = loadDataSet('testSetRBF.txt') 
 b,alphas = smoP(dataArr, labelArr, 200, 0.0001, 10000, ('rbf', k1)) #C=200 important 
 datMat=mat(dataArr); labelMat = mat(labelArr).transpose() 
 svInd=nonzero(alphas.A>0)[0] 
 sVs=datMat[svInd] #get matrix of only support vectors 
 labelSV = labelMat[svInd]; 
 print "there are %d Support Vectors" % shape(sVs)[0] 
 m,n = shape(datMat) 
 errorCount = 0 
 for i in range(m): 
  kernelEval = kernelTrans(sVs,datMat[i,:],('rbf', k1)) 
  predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b 
  if sign(predict)!=sign(labelArr[i]): errorCount += 1 
 print "the training error rate is: %f" % (float(errorCount)/m) 
 dataArr,labelArr = loadDataSet('testSetRBF2.txt') 
 errorCount = 0 
 datMat=mat(dataArr); labelMat = mat(labelArr).transpose() 
 m,n = shape(datMat) 
 for i in range(m): 
  kernelEval = kernelTrans(sVs,datMat[i,:],('rbf', k1)) 
  predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b 
  if sign(predict)!=sign(labelArr[i]): errorCount += 1  
 print "the test error rate is: %f" % (float(errorCount)/m)  
  
def img2vector(filename): 
 returnVect = zeros((1,1024)) 
 fr = open(filename) 
 for i in range(32): 
  lineStr = fr.readline() 
  for j in range(32): 
   returnVect[0,32*i+j] = int(lineStr[j]) 
 return returnVect 
 
def loadImages(dirName): 
 from os import listdir 
 hwLabels = [] 
 trainingFileList = listdir(dirName)   #load the training set 
 m = len(trainingFileList) 
 trainingMat = zeros((m,1024)) 
 for i in range(m): 
  fileNameStr = trainingFileList[i] 
  fileStr = fileNameStr.split('.')[0]  #take off .txt 
  classNumStr = int(fileStr.split('_')[0]) 
  if classNumStr == 9: hwLabels.append(-1) 
  else: hwLabels.append(1) 
  trainingMat[i,:] = img2vector('%s/%s' % (dirName, fileNameStr)) 
 return trainingMat, hwLabels  
 
def testDigits(kTup=('rbf', 10)): 
 dataArr,labelArr = loadImages('trainingDigits') 
 b,alphas = smoP(dataArr, labelArr, 200, 0.0001, 10000, kTup) 
 datMat=mat(dataArr); labelMat = mat(labelArr).transpose() 
 svInd=nonzero(alphas.A>0)[0] 
 sVs=datMat[svInd] 
 labelSV = labelMat[svInd]; 
 print "there are %d Support Vectors" % shape(sVs)[0] 
 m,n = shape(datMat) 
 errorCount = 0 
 for i in range(m): 
  kernelEval = kernelTrans(sVs,datMat[i,:],kTup) 
  predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b 
  if sign(predict)!=sign(labelArr[i]): errorCount += 1 
 print "the training error rate is: %f" % (float(errorCount)/m) 
 dataArr,labelArr = loadImages('testDigits') 
 errorCount = 0 
 datMat=mat(dataArr); labelMat = mat(labelArr).transpose() 
 m,n = shape(datMat) 
 for i in range(m): 
  kernelEval = kernelTrans(sVs,datMat[i,:],kTup) 
  predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b 
  if sign(predict)!=sign(labelArr[i]): errorCount += 1  
 print "the test error rate is: %f" % (float(errorCount)/m) 
 
 
'''''#######******************************** 
Non-Kernel VErsions below 
'''#######******************************** 
 
class optStructK: 
 def __init__(self,dataMatIn, classLabels, C, toler): # Initialize the structure with the parameters 
  self.X = dataMatIn 
  self.labelMat = classLabels 
  self.C = C 
  self.tol = toler 
  self.m = shape(dataMatIn)[0] 
  self.alphas = mat(zeros((self.m,1))) 
  self.b = 0 
  self.eCache = mat(zeros((self.m,2))) #first column is valid flag 
   
def calcEkK(oS, k): 
 fXk = float(multiply(oS.alphas,oS.labelMat).T*(oS.X*oS.X[k,:].T)) + oS.b 
 Ek = fXk - float(oS.labelMat[k]) 
 return Ek 
   
def selectJK(i, oS, Ei):   #this is the second choice -heurstic, and calcs Ej 
 maxK = -1; maxDeltaE = 0; Ej = 0 
 oS.eCache[i] = [1,Ei] #set valid #choose the alpha that gives the maximum delta E 
 validEcacheList = nonzero(oS.eCache[:,0].A)[0] 
 if (len(validEcacheList)) > 1: 
  for k in validEcacheList: #loop through valid Ecache values and find the one that maximizes delta E 
   if k == i: continue #don't calc for i, waste of time 
   Ek = calcEk(oS, k) 
   deltaE = abs(Ei - Ek) 
   if (deltaE > maxDeltaE): 
    maxK = k; maxDeltaE = deltaE; Ej = Ek 
  return maxK, Ej 
 else: #in this case (first time around) we don't have any valid eCache values 
  j = selectJrand(i, oS.m) 
  Ej = calcEk(oS, j) 
 return j, Ej 
 
def updateEkK(oS, k):#after any alpha has changed update the new value in the cache 
 Ek = calcEk(oS, k) 
 oS.eCache[k] = [1,Ek] 
   
def innerLK(i, oS): 
 Ei = calcEk(oS, i) 
 if ((oS.labelMat[i]*Ei < -oS.tol) and (oS.alphas[i] < oS.C)) or ((oS.labelMat[i]*Ei > oS.tol) and (oS.alphas[i] > 0)): 
  j,Ej = selectJ(i, oS, Ei) #this has been changed from selectJrand 
  alphaIold = oS.alphas[i].copy(); alphaJold = oS.alphas[j].copy(); 
  if (oS.labelMat[i] != oS.labelMat[j]): 
   L = max(0, oS.alphas[j] - oS.alphas[i]) 
   H = min(oS.C, oS.C + oS.alphas[j] - oS.alphas[i]) 
  else: 
   L = max(0, oS.alphas[j] + oS.alphas[i] - oS.C) 
   H = min(oS.C, oS.alphas[j] + oS.alphas[i]) 
  if L==H: print "L==H"; return 0 
  eta = 2.0 * oS.X[i,:]*oS.X[j,:].T - oS.X[i,:]*oS.X[i,:].T - oS.X[j,:]*oS.X[j,:].T 
  if eta >= 0: print "eta>=0"; return 0 
  oS.alphas[j] -= oS.labelMat[j]*(Ei - Ej)/eta 
  oS.alphas[j] = clipAlpha(oS.alphas[j],H,L) 
  updateEk(oS, j) #added this for the Ecache 
  if (abs(oS.alphas[j] - alphaJold) < 0.00001): print "j not moving enough"; return 0 
  oS.alphas[i] += oS.labelMat[j]*oS.labelMat[i]*(alphaJold - oS.alphas[j])#update i by the same amount as j 
  updateEk(oS, i) #added this for the Ecache     #the update is in the oppostie direction 
  b1 = oS.b - Ei- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.X[i,:]*oS.X[i,:].T - oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.X[i,:]*oS.X[j,:].T 
  b2 = oS.b - Ej- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.X[i,:]*oS.X[j,:].T - oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.X[j,:]*oS.X[j,:].T 
  if (0 < oS.alphas[i]) and (oS.C > oS.alphas[i]): oS.b = b1 
  elif (0 < oS.alphas[j]) and (oS.C > oS.alphas[j]): oS.b = b2 
  else: oS.b = (b1 + b2)/2.0 
  return 1 
 else: return 0 
 
def smoPK(dataMatIn, classLabels, C, toler, maxIter): #full Platt SMO 
 oS = optStruct(mat(dataMatIn),mat(classLabels).transpose(),C,toler) 
 iter = 0 
 entireSet = True; alphaPairsChanged = 0 
 while (iter < maxIter) and ((alphaPairsChanged > 0) or (entireSet)): 
  alphaPairsChanged = 0 
  if entireSet: #go over all 
   for i in range(oS.m):   
    alphaPairsChanged += innerL(i,oS) 
    print "fullSet, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  else:#go over non-bound (railed) alphas 
   nonBoundIs = nonzero((oS.alphas.A > 0) * (oS.alphas.A < C))[0] 
   for i in nonBoundIs: 
    alphaPairsChanged += innerL(i,oS) 
    print "non-bound, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  if entireSet: entireSet = False #toggle entire set loop 
  elif (alphaPairsChanged == 0): entireSet = True 
  print "iteration number: %d" % iter 
 return oS.b,oS.alphas

运行结果如(图八)所示:

python机器学习理论与实战(六)支持向量机

(图八)

上面代码有兴趣的可以读读,用的话,建议使用libsvm。

参考文献:

    [1]machine learning in action. PeterHarrington

    [2] pattern recognition and machinelearning. Christopher M. Bishop

    [3]machine learning.Andrew Ng

以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持三水点靠木。

Python 相关文章推荐
Python实现简单过滤文本段的方法
May 24 Python
机器学习之KNN算法原理及Python实现方法详解
Jul 09 Python
python cs架构实现简单文件传输
Mar 20 Python
tensorflow实现加载mnist数据集
Sep 08 Python
django将网络中的图片,保存成model中的ImageField的实例
Aug 07 Python
python xlwt如何设置单元格的自定义背景颜色
Sep 03 Python
python实现批量文件重命名
Oct 31 Python
40行Python代码实现天气预报和每日鸡汤推送功能
Feb 27 Python
Python ArgumentParse的subparser用法说明
Apr 20 Python
python virtualenv虚拟环境配置与使用教程详解
Jul 13 Python
如何利用python检测图片是否包含二维码
Oct 15 Python
python+appium+yaml移动端自动化测试框架实现详解
Nov 24 Python
Python logging管理不同级别log打印和存储实例
Jan 19 #Python
python机器学习理论与实战(五)支持向量机
Jan 19 #Python
Python读取图片为16进制表示简单代码
Jan 19 #Python
Python实现pdf文档转txt的方法示例
Jan 19 #Python
浅谈Python实现2种文件复制的方法
Jan 19 #Python
用Python进行简单图像识别(验证码)
Jan 19 #Python
flask中使用蓝图将路由分开写在不同文件实例解析
Jan 19 #Python
You might like
PHP新手上路(八)
2006/10/09 PHP
PHP添加图片水印、压缩、剪切的封装类
2015/08/17 PHP
ThinkPHP实现生成和校验验证码功能
2017/04/28 PHP
js 替换功能函数,用正则表达式解决,js的全部替换
2010/12/08 Javascript
JavaScript中的this到底是什么(一)
2015/12/09 Javascript
基于JS实现checkbox全选功能实例代码
2016/10/31 Javascript
微信小程序通过api接口将json数据展现到小程序示例
2017/01/20 Javascript
Bootstrap输入框组件简单实现代码
2017/03/06 Javascript
Vue中this.$router.push参数获取方法
2018/02/27 Javascript
深入理解NodeJS 多进程和集群
2018/10/17 NodeJs
vue中$refs, $emit, $on, $once, $off的使用详解
2019/05/26 Javascript
jQuery实现图片下载代码
2019/07/18 jQuery
vue中jsonp插件的使用方法示例
2020/09/10 Javascript
jQuery实现鼠标拖拽登录框移动效果
2020/09/13 jQuery
Vuex实现简单购物车
2021/01/10 Vue.js
[51:20]完美世界DOTA2联赛PWL S2 Magma vs PXG 第一场 11.28
2020/12/01 DOTA
python类参数self使用示例
2014/02/17 Python
Python 正则表达式入门(初级篇)
2016/12/07 Python
Python部署web开发程序的几种方法
2017/05/05 Python
快速了解Python中的装饰器
2018/01/11 Python
Python切片操作深入详解
2018/07/27 Python
深入flask之异步非堵塞实现代码示例
2018/07/31 Python
django+echart绘制曲线图的方法示例
2018/11/26 Python
python nmap实现端口扫描器教程
2020/05/28 Python
Python字典生成式、集合生成式、生成器用法实例分析
2020/01/07 Python
Python 动态变量名定义与调用方法
2020/02/09 Python
使用python无账号无限制获取企查查信息的实例代码
2020/04/17 Python
纯CSS3实现表单验证效果(非常不错)
2017/01/18 HTML / CSS
CSS3条纹背景制作的实战攻略
2016/05/31 HTML / CSS
台湾旅游网站:雄狮旅游网
2017/08/16 全球购物
励志演讲稿600字
2014/08/21 职场文书
政府领导干部个人对照检查材料思想汇报
2014/09/24 职场文书
字典算法实现及操作 --python(实用)
2021/03/31 Python
Python 中的单分派泛函数你真的了解吗
2021/06/22 Python
sql注入教程之类型以及提交注入
2021/08/02 MySQL
CSS实现鼠标悬浮动画特效
2023/05/07 HTML / CSS