人工智能1

一logistic回归

1.阅读并运行程序

程序阅读

% 课堂演示-- Logistic Regression

%本例子的说明和代码来自：

https://www.360docs.net/doc/555820350.html,/MainFolder/DocumentPage.php?course=M achineLearning&doc=exercises/ex4/ex4.html

clear all; close all; clc %清空所有窗口和历史记录；关闭窗口

x = load('ex4x.dat') %从ex4x.dat中下载x的数据

y = load('ex4y.dat') %从ex4y.dat中下载y的数据

[m, n] = size(x) %x为一个m*n的矩阵，m为80，n为40

% Add intercept term to x

x = [ones(m, 1), x] %将m置1再将其与x中的所有数据放到x中去

% Plot the training data

% Use different markers for positives and negatives

figure

pos = find(y) %找到y是符合条件

neg = find(y == 0) %如果y等于0，则表示不符合条件

plot(x(pos, 2), x(pos,3), '+')

hold on

hold on

xlabel('Exam 1 score') %测试1的成绩,并把Exam 1 score的值设置为x 轴

ylabel('Exam 2 score') %测试2的成绩，并把Exam 1 score的值设置为y 轴

% Initialize fitting parameters

theta = zeros(n+1, 1); %x这个矩阵中第n列加1，和1这两个数置零，也就是说两个不同的样本分成两个阵列输出

% Define the sigmoid function

g = inline('1.0 ./ (1.0 + exp(-z))'); %拟合函数，实现回归问题（1/(1+e-^x)），拟合一条成本函数曲线

% Newton's method

MAX_ITR = 7; %利用牛顿迭代法进行模拟，最多可迭代7次

J = zeros(MAX_ITR, 1);

for i = 1:MAX_ITR %开始迭代计算拟合曲线，从1到7

% Calculate the hypothesis function

z = x * theta; %所有x诚意theta，赋给z这个变量

h = g(z); %调用拟合函数，拟合一条曲线，用y表示

% Calculate gradient and hessian.

% The formulas below are equivalent to the summation formulas

% given in the lecture videos.

grad = (1/m).*x' * (h-y); %计算下降梯度

H = (1/m).*x' * diag(h) * diag(1-h) * x;

% Calculate J (for testing convergence)

J(i) =(1/m)*sum(-y.*log(h) - (1-y).*log(1-h));

%计算梯度

theta = theta - H\grad; %theta的值

end

% Display theta

theta

% Calculate the probability that a student with

% Score 20 on exam 1 and score 80 on exam 2

% will not be admitted

prob = 1 - g([1, 20, 80]*theta)%求theta的最终函数值

% Plot Newton's method result

% Only need 2 points to define a line, so choose two endpoints

%找出x所有行中第二列的数据中的最小数据，减去2；

%以及找出x所有行中第二列的数据中的最大数据，加上2，放到plot_x中，置一条直线。

%（找到两个点，一个大点，一个小点，然后画一条直线），执行下面的语句plot_x = [min(x(:,2))-2, max(x(:,2))+2];

% Calculate the decision boundary line

%计算并确定这一条分界线，然后将其画出来，执行下面的语句

plot_y = (-1./theta(3)).*(theta(2).*plot_x +theta(1));

plot(plot_x, plot_y) %在一个二维空间坐标中画这条分界线

legend('Admitted', 'Not admitted', 'Decision Boundary')

hold off

% Plot J

figure

%画图，取8个点，并且O中用红色填充

plot(0:MAX_ITR-1, J, 'o--', 'MarkerFaceColor', 'r', 'MarkerSize', 8)

xlabel('Iteration'); ylabel('J') %对其进行迭代，找到最优的解

% Display J

J

运行截图：

图1

图2

theta =

-16.3787

0.1483

0.1589

prob =

0.6680

Warning: Ignoring extra legend entries.

> In legend at 239

In logistic_regression at 70

J =

0.6931

0.4409

0.4089

0.4055

0.4054

0.4054

0.4054

2.将程序求theta的迭代部分改成梯度下降法（参见课件），程序中采用的是牛顿法，该方法的特点是收敛速度较梯度下降法更快。

改进后的程序：

% 课堂演示-- Logistic Regression

%本例子的说明和代码来自：

https://www.360docs.net/doc/555820350.html,/MainFolder/DocumentPage.php?course=M achineLearning&doc=exercises/ex4/ex4.html

clear all; close all; clc %清空所有窗口和历史记录；关闭窗口

x = load('ex4x.dat') %从ex4x.dat中下载x的数据

y = load('ex4y.dat') %从ex4y.dat中下载y的数据

[m, n] = size(x) %x为一个m*n的矩阵，m为80，n为40

% Add intercept term to x

x = [ones(m, 1), x] %将m置1再将其与x中的所有数据放到x中去

% Plot the training data

% Use different markers for positives and negatives

figure

pos = find(y) %找到y是符合条件

neg = find(y == 0) %如果y等于0，则表示不符合条件

plot(x(pos, 2), x(pos,3), '*')

hold on

hold on

xlabel('Exam 1 score') %测试1的成绩,并把Exam 1 score的值设置为x 轴

ylabel('Exam 2 score') %测试2的成绩，并把Exam 1 score的值设置为y 轴

% Initialize fitting parameters

%theta = zeros(n+1, 1); %x这个矩阵中第n列加1，和1这两个数置零，也就是说两个不同的样本分成两个阵列输出

theta=[-8;0.8;0.5] %给theta赋初值

% Define the sigmoid function

g = inline('1.0 ./ (1.0 + exp(-z))'); %拟合函数，实现回归问题（1/(1+e-^x)），拟合一条成本函数曲线

% 梯度下降法

MAX_ITR =5000; %利用梯度下降迭代法进行模拟，最多可迭代500次

alpha=0.0001;

J = zeros(MAX_ITR, 1);

for i = 1:MAX_ITR %开始迭代计算拟合曲线，从1到7 % Calculate the hypothesis function

z = x * theta; %所有x诚意theta，赋给z这个变量

h = g(z); %调用拟合函数，拟合一条曲线，用y表示

% Calculate gradient and hessian.

% The formulas below are equivalent to the summation formulas % given in the lecture videos.

grad = (1/m).*x' * (h-y); %计算下降梯度

%H = (1/m).*x' * diag(h) * diag(1-h) * x;

% Calculate J (for testing convergence)

theta = theta-alpha.* grad;

J(i) =(1/m)*sum(-y.*log(h) - (1-y).*log(1-h));

end

% Display theta

theta

% Calculate the probability that a student with

% Score 20 on exam 1 and score 80 on exam 2

% will not be admitted

prob = 1 - g([1, 20, 80]*theta)%求theta的最终函数值

% Plot Newton's method result

% Only need 2 points to define a line, so choose two endpoints

%找出x所有行中第二列的数据中的最小数据，减去2；

%以及找出x所有行中第二列的数据中的最大数据，加上2，放到plot_x中，置一条直线。

%（找到两个点，一个大点，一个小点，然后画一条直线），执行下面的语句plot_x = [min(x(:,2))-2, max(x(:,2))+2];

% Calculate the decision boundary line

%计算并确定这一条分界线，然后将其画出来，执行下面的语句

plot_y = (-1./theta(3)).*(theta(2).*plot_x +theta(1));

plot(plot_x, plot_y) %在一个二维空间坐标中画这条分界线

legend('Admitted', 'Not admitted', 'Decision Boundary')

hold off

% Plot J

figure

%画图，取8个点，并且O中用蓝色填充

plot(0:MAX_ITR-1, J, 'o--', 'MarkerFaceColor', 'g', 'MarkerSize', 8)

xlabel('Iteration'); ylabel('J') %对其进行迭代，找到最优的解

% Display J

J

运行截图

二线性回归：

1.请阅读、运行程序

% Exercise 2 Linear Regression

% 资料源自：

https://www.360docs.net/doc/555820350.html,/MainFolder/DocumentPage.php?course=M achineLearning&doc=exercises/ex2/ex2.html

% Data is roughly based on 2000 CDC growth figures

% for boys

%

% x refers to a boy's age

% y is a boy's height in meters

%

clear all; close all; clc %清空所有记录和关闭之前打开的所有窗口

x = load('ex2x.dat'); %从ex2x.da这个数据包中下载数据到x

y = load('ex2y.dat'); %从ex2y.dat这个数据包中下载数据在y

m = length(y); % number of training examples

%定义训练的样本（y）的长度或大小

% Plot the training data（开始训练函数）

figure; % open a new figure window（打开一个新的窗口）plot(x, y, 'o'); %定义输出样本用O表示

ylabel('Height in meters') %定义y轴的值为身高

xlabel('Age in years') %定义x轴的值为年龄

% Gradient descent

x = [ones(m, 1) x] % Add a column of ones to x（添加到x 的那些列：就是将m设为1的那些列）

theta = zeros(size(x(1,:)))'% initialize fitting parameters（对theta进行初始化）

%将x中第一行和所有列初始化

MAX_ITR = 1500; %训练或者是迭代的最大次数为1500次alpha = 0.07; %定义学习率为0.07

for num_iterations = 1:MAX_ITR %梯度下降发开始，对回归函数开始迭代，从1到1500

% This is a vectorized version of the

% gradient descent update formula

% It's also fine to use the summation formula from the videos

% Here is the gradient

grad = (1/m).* x' * ((x * theta) - y); %梯度下降发函数实现语句

% Here is the actual update

theta = theta - alpha .* grad; %计算theta的值

% Sequential update: The wrong way to do gradient descent

% grad1 = (1/m).* x(:,1)' * ((x * theta) - y);

% theta(1) = theta(1) + alpha*grad1;

% grad2 = (1/m).* x(:,2)' * ((x * theta) - y);

% theta(2) = theta(2) + alpha*grad2;

end

% print theta to screen

theta

% Plot the linear fit

hold on; % keep previous plot visible

plot(x(:,2), x*theta, '-') %寻找一个最低点，并将其输出来legend('Training data', 'Linear regression')%对数据进行训练

hold off % don't overlay any more plots on this figure

% Closed form solution for reference

% You will learn about this method in future videos

exact_theta = (x' * x)\x' * y %确定theta值

% Predict values for age 3.5 and 7

predict1 = [1, 3.5] *theta %预测1米，3.5岁的个体是否符合训练样本的规律

predict2 = [1, 7] * theta %预测1米，7岁的个体是否符合训练样本的规律

% Calculate J matrix

% Grid over which we will calculate J

theta0_vals = linspace(-3, 3, 100); %theta0的值为-3到3，并分割为100等分，并将其用等高线输出

theta1_vals = linspace(-1, 1, 100); %theta1的值为-1到1，并分割为100等分

% initialize J_vals to a matrix of 0's

J_vals = zeros(length(theta0_vals), length(theta1_vals))

for i = 1:length(theta0_vals) %开始迭代，i从1到1500 for j = 1:length(theta1_vals) %开始迭代，j从1到1500

t = [theta0_vals(i); theta1_vals(j)];

J_vals(i,j) = (0.5/m) .* (x * t - y)' * (x * t - y);

end

end

% Because of the way meshgrids work in the surf command, we need to

% transpose J_vals before calling surf, or else the axes will be flipped

J_vals = J_vals'; %将J_vals'矩阵付给J_vals

% Surface plot

figure;

surf(theta0_vals, theta1_vals, J_vals) %利用surf函数求theta0_vals、theta1_vals、J_vals的值

xlabel('\theta_0'); ylabel('\theta_1'); %x轴表示theta_0，y轴表示theta_1

% Contour plot

figure;

% Plot J_vals as 15 contours spaced logarithmically between 0.01 and 100 contour(theta0_vals, theta1_vals, J_vals, logspace(-2, 2, 15))%输出梯度下降中的最低点，用等高线表示

xlabel('\theta_0'); ylabel('\theta_1'); %x轴表示theta_0，y轴表示theta_1

运行截图

图1

图2

图3

训练数据

theta =

theta =

0.7502

0.0639

exact_theta =

0.7502

0.0639

predict1 =

0.9737

predict2 =

1.1973

J_vals =

Columns 1 through 15

2.手工产生训练数据。训练数据除了现有的年龄外，再增加一个“体重”因子，在做线性回归

% Exercise 2 Linear Regression

% 资料源自：

https://www.360docs.net/doc/555820350.html,/MainFolder/DocumentPage.php?course=M achineLearning&doc=exercises/ex2/ex2.html

% Data is roughly based on 2000 CDC growth figures

% for boys

%

% x refers to a boy's age

% y is a boy's height in meters

%

clear all; close all; clc %清空所有记录和关闭之前打开的所有窗口

x1 = load('ex2x.dat'); %从ex2x.da这个数据包中下载数据到x

y = load('ex2y.dat'); %从ex2y.dat这个数据包中下载数据在y

x2=[31;32;33;34;35;36;37;38;39;40;41;42;43;44;45;46;47;48;49;50];

%初始化x2，设置为体重，单位kg

m = length(y); % number of training examples

%定义训练的样本（y）的长度或大小

% Plot the training data（开始训练函数）

figure; % open a new figure window（打开一个新的窗口）plot3(x1, x2, y, 'o'); %定义输出样本用O表示

ylabel('Height in meters') %定义y轴的值为身高

x1label('Age in years') %定义x1轴的值为年龄

x2label('weight in kilogram')%定义x2轴的值为体重

% Gradient descent

x = [ones(m, 1), x1,x2] % Add a column of ones to x（添加到x1,x2的那些列：就是将m设为1的那些列）

theta = zeros(size(x(1,:)))'% initialize fitting parameters（对theta进行初始化）

%将x中第一行和所有列初始化

MAX_ITR = 5000; %训练或者是迭代的最大次数为5000次

alpha = 0.005; %定义学习率为0.005

for num_iterations = 1:MAX_ITR %梯度下降发开始，对回归函数开始迭代，从1到5000

% This is a vectorized version of the

% gradient descent update formula

% It's also fine to use the summation formula from the videos

% Here is the gradient

grad = (1/m).* x' * ((x * theta) - y);%梯度下降发函数实现语句

% Here is the actual update

theta = theta - alpha .* grad; %计算theta的值

% Sequential update: The wrong way to do gradient descent

% grad1 = (1/m).* x(:,1)' * ((x * theta) - y);

% theta(1) = theta(1) + alpha*grad1;

% grad2 = (1/m).* x(:,2)' * ((x * theta) - y);

% theta(2) = theta(2) + alpha*grad2;

end

% print theta to screen

theta

% Plot the linear fit

hold on; % keep previous plot visible

plot(x(:,[2,3]), x*theta, 'r-') %寻找一个最低点，并将其输出来legend('Training data', 'Linear regression')%对数据进行训练

hold off % don't overlay any more plots on this figure

% Closed form solution for reference

% You will learn about this method in future videos

exact_theta = (x' * x)\(x' * y ) %确定theta值

% Predict values for age 3.5 and 7

predict1 = [1, 4.5 ,45] *theta %预测1米，4.5岁，45kg是否符合训练样本的规律

predict2 = [1, 7,35] * theta %预测1米，7岁,, 45kg否符合训练样本的规律% Calculate J matrix

% Grid over which we will calculate J

theta0_vals = linspace(-3, 3, 100); %theta0的值为-3到3，并分割为100等分，并将其用等高线输出

theta1_vals = linspace(-1, 1, 100); %theta1的值为-1到1，并分割为100等分

% initialize J_vals to a matrix of 0's

J_vals = zeros(length(theta0_vals), length(theta1_vals)，length(theta2_vals)) for i = 1:length(theta0_vals) %开始迭代，i从1到1500 for j = 1:length(theta1_vals) %开始迭代，j从1到1500

t = [theta0_vals(i); theta1_vals(j)];

J_vals(i,j,k) = (0.5/m) .* (x * t - y)' * (x * t - y);

end

end

% Because of the way meshgrids work in the surf command, we need to

% transpose J_vals before calling surf, or else the axes will be flipped

J_vals = J_vals'; %将J_vals'矩阵付给J_vals

% Surface plot

figure;

surf(theta0_vals, theta1_vals, J_vals) %利用surf函数求theta0_vals、theta1_vals, J_vals的值

xlabel('\theta_0'); ylabel('\theta_1'); %x轴表示theta_0，y轴表示theta_1,z轴表示theta_2

% Contour plot

figure;

% Plot J_vals as 15 contours spaced logarithmically between 0.01 and 100 contour(theta0_vals, theta1_vals, J_vals, logspace(-2, 2, 15))%输出梯度下降中的最低点，用等高线表示

xlabel('\theta_0'); ylabel('\theta_1'); %x轴表示theta_0，y轴表示theta_1 二部完成后，对新数据进行预测

% Exercise 2 Linear Regression

% 资料源自：

https://www.360docs.net/doc/555820350.html,/MainFolder/DocumentPage.php?course=M achineLearning&doc=exercises/ex2/ex2.html

% Data is roughly based on 2000 CDC growth figures

% for boys

%

% x refers to a boy's age

% y is a boy's height in meters

%

clear all; close all; clc %清空所有记录和关闭之前打开的所有窗口

x1 = load('ex2x.dat'); %从ex2x.da这个数据包中下载数据到x

y = load('ex2y.dat'); %从ex2y.dat这个数据包中下载数据在y

x2 = load('ex2z.dat'); %从ex2z.dat这个数据包中下载数据在y

%x2=[31;32;33;34;35;36;37;38;39;40;41;42;43;44;45;46;47;48;49;50];

%初始化x2，设置为体重，单位kg

m = length(y); % number of training examples

%定义训练的样本（y）的长度或大小

% Plot the training data（开始训练函数）

figure; % open a new figure window（打开一个新的窗口）plot3(x1, x2, y, 'o'); %定义输出样本用O表示

ylabel('Height in meters') %定义y轴的值为身高

x1label('Age in years') %定义x1轴的值为年龄

x2label('weight in kilogram')%定义x2轴的值为体重

% Gradient descent

x = [ones(m, 1), x1,x2] % Add a column of ones to x（添加到x1,x2的那些列：就是将m设为1的那些列）

theta = zeros(size(x(1,:)))'% initialize fitting parameters（对theta进行初始化）

%将x中第一行和所有列初始化

MAX_ITR = 5000; %训练或者是迭代的最大次数为5000次

alpha = 0.005; %定义学习率为0.005

for num_iterations = 1:MAX_ITR %梯度下降发开始，对回归函数开始迭代，从1到5000

% This is a vectorized version of the

% gradient descent update formula

% It's also fine to use the summation formula from the videos

% Here is the gradient

grad = (1/m).* x' * ((x * theta) - y);%梯度下降发函数实现语句

% Here is the actual update

theta = theta - alpha .* grad; %计算theta的值

% Sequential update: The wrong way to do gradient descent

% grad1 = (1/m).* x(:,1)' * ((x * theta) - y);

% theta(1) = theta(1) + alpha*grad1;

% grad2 = (1/m).* x(:,2)' * ((x * theta) - y);

% theta(2) = theta(2) + alpha*grad2;

end

% print theta to screen

theta

% Plot the linear fit

hold on; % keep previous plot visible

plot(x(:,[2,3]), x*theta, 'r-') %寻找一个最低点，并将其输出来legend('Training data', 'Linear regression')%对数据进行训练

hold off % don't overlay any more plots on this figure