[Watermelon_book] Chapter 3 Linear Model

Linear Model
- 基本定义
- 线性模型简单形式的实际编码
- - Task
  - Generate data
  - Cost function
  - Gradient descent
  - Training
  - Model evaluation
  - - Experiment 1
    - Experiment 2
  - Task 2
  - Generate data
  - Normalization
  - Prediction format
  - Cost function
  - Gradient descent
  - Training
  - Model evaluation
- 由回归到分类
- Logistic Regression ( Binary Classification）的实际编码
- - Dataset
  - Visulization
  - Prediction Format
  - Cost function
  - Gradient descent
  - Training
  - Model evaluation
  - - Visulization
    - Mapping probabilities to classes

Linear Model

基本定义

线性模型简单形式的实际编码

Task

We have derived the formula of Linear Model, here I want to write codes for it by hand. The specific task is to predict sales based on radios shown as the table below.
The original problem is from https://ml-cheatsheet.readthedocs.io/en/latest/linear_regression.html

Company	Radio($)	Sales
Amazon	37.8	22.1
Google	39.3	10.4

Generate data

import numpy as np
import scipy
import seaborn as sns
import matplotlib.pyplot as plt

sample_size = 200
np.random.seed(5)
radio_sample = 60 * np.random.rand(sample_size)
weight_truth = 0.4
bias_truth = -3
np.random.seed(10)
nosie_sample = 10 * (np.random.normal(0, 0.1, sample_size))
sales_sample = radio_sample * weight_truth + bias_truth + nosie_sample
sales_no_noise = radio_sample * weight_truth + bias_truth

plt.scatter(radio_sample, sales_sample, alpha=0.5)
plt.plot(radio_sample, sales_no_noise, c="r")
plt.xlabel("radio")
plt.ylabel("sales")

Text(0, 0.5, 'sales')

Cost function

$b)=\frac{1}{N} \sum_{i=1}^{n}\left(y_{i}-\left(m x_{i}+b\right)\right)^{2}$

def cost_function(radio, sales, weight, bias):
    '''cost_function for linear model
    Args: 
        radio,sales: is numpy array
        weight,bias: is scalar
    '''
    sample_size = len(radio)
    error = 0.0
    for i in range(sample_size):
        error += (sales[i] - (radio[i]*weight + bias))**2
    error_avg = error/sample_size
    return error_avg

Gradient descent

\begin{aligned} f^{\prime}(m, b)=\left[ \begin{array}{c}{\frac{d f}{d m}} \ {\frac{d f}{d b}}\end{array}\right] &=\left[ \begin{array}{c}{\frac{1}{N} \sum-2 x_{i}\left(y_{i}-\left(m x_{i}+b\right)\right)} \ {\frac{1}{N} \sum-2\left(y_{i}-\left(m x_{i}+b\right)\right)}\end{array}\right] \end{aligned}

def update_weight(radio, sales, weight, bias, learning_rate):
    # initial value
    weight_deriv = 0
    bias_deriv = 0
    sample_size = len(radio)
    
    for i in range(sample_size):
        # calculate partial derivatives
        weight_deriv += -2*radio[i]*(sales[i] - (weight*radio[i] + bias))
        bias_deriv += -2*(sales[i] - (weight*radio[i] + bias))
    
    #gradient descent
    weight = weight - (weight_deriv/sample_size)*learning_rate
    bias = bias - (bias_deriv/sample_size)*learning_rate
    
    return weight,bias

Training

def train(radio, sales, weight, bias, learning_rate, iters):
    cost_history = []
    for i in range(iters):
        weight, bias = update_weight(radio, sales, weight, bias, learning_rate)
        cost = cost_function(radio, sales, weight, bias)
        cost_history.append(cost)
        if i % 5 == 0:
            print("Iter: %d \t Weight: %2f \t Bias: %2f \t Cost: %4f" %(i,weight, bias, cost))
    return weight, bias, cost_history

Model evaluation

Experiment 1

weight_final, bias_final, cost_history = train(radio_sample, sales_sample, 0, 0, 0.0001, 2000)

Iter: 0 	 Weight: 0.080657 	 Bias: 0.001851 	 Cost: 77.703045
Iter: 5 	 Weight: 0.267103 	 Bias: 0.005752 	 Cost: 7.320940
Iter: 10 	 Weight: 0.312288 	 Bias: 0.006165 	 Cost: 3.188519
Iter: 15 	 Weight: 0.323249 	 Bias: 0.005734 	 Cost: 2.945016
Iter: 20 	 Weight: 0.325918 	 Bias: 0.005099 	 Cost: 2.929796
Iter: 25 	 Weight: 0.326578 	 Bias: 0.004414 	 Cost: 2.927978
Iter: 30 	 Weight: 0.326751 	 Bias: 0.003717 	 Cost: 2.926946
Iter: 35 	 Weight: 0.326806 	 Bias: 0.003017 	 Cost: 2.925961
Iter: 40 	 Weight: 0.326832 	 Bias: 0.002317 	 Cost: 2.924980
Iter: 45 	 Weight: 0.326852 	 Bias: 0.001617 	 Cost: 2.923999
Iter: 50 	 Weight: 0.326869 	 Bias: 0.000917 	 Cost: 2.923018
Iter: 55 	 Weight: 0.326887 	 Bias: 0.000217 	 Cost: 2.922038
Iter: 60 	 Weight: 0.326904 	 Bias: -0.000482 	 Cost: 2.921058
Iter: 65 	 Weight: 0.326921 	 Bias: -0.001182 	 Cost: 2.920079
Iter: 70 	 Weight: 0.326939 	 Bias: -0.001881 	 Cost: 2.919101
Iter: 75 	 Weight: 0.326956 	 Bias: -0.002580 	 Cost: 2.918123
Iter: 80 	 Weight: 0.326973 	 Bias: -0.003279 	 Cost: 2.917145
Iter: 85 	 Weight: 0.326990 	 Bias: -0.003978 	 Cost: 2.916168
Iter: 90 	 Weight: 0.327008 	 Bias: -0.004677 	 Cost: 2.915191
Iter: 95 	 Weight: 0.327025 	 Bias: -0.005375 	 Cost: 2.914215
Iter: 100 	 Weight: 0.327042 	 Bias: -0.006073 	 Cost: 2.913240
Iter: 105 	 Weight: 0.327059 	 Bias: -0.006771 	 Cost: 2.912264
Iter: 110 	 Weight: 0.327077 	 Bias: -0.007469 	 Cost: 2.911290
Iter: 115 	 Weight: 0.327094 	 Bias: -0.008167 	 Cost: 2.910315
Iter: 120 	 Weight: 0.327111 	 Bias: -0.008864 	 Cost: 2.909342
Iter: 125 	 Weight: 0.327128 	 Bias: -0.009562 	 Cost: 2.908368
Iter: 130 	 Weight: 0.327145 	 Bias: -0.010259 	 Cost: 2.907396
Iter: 135 	 Weight: 0.327163 	 Bias: -0.010956 	 Cost: 2.906423
Iter: 140 	 Weight: 0.327180 	 Bias: -0.011653 	 Cost: 2.905452
Iter: 145 	 Weight: 0.327197 	 Bias: -0.012350 	 Cost: 2.904480
Iter: 150 	 Weight: 0.327214 	 Bias: -0.013046 	 Cost: 2.903509
Iter: 155 	 Weight: 0.327231 	 Bias: -0.013742 	 Cost: 2.902539
Iter: 160 	 Weight: 0.327249 	 Bias: -0.014439 	 Cost: 2.901569
Iter: 165 	 Weight: 0.327266 	 Bias: -0.015135 	 Cost: 2.900600
Iter: 170 	 Weight: 0.327283 	 Bias: -0.015830 	 Cost: 2.899631
Iter: 175 	 Weight: 0.327300 	 Bias: -0.016526 	 Cost: 2.898663
Iter: 180 	 Weight: 0.327317 	 Bias: -0.017222 	 Cost: 2.897695
Iter: 185 	 Weight: 0.327335 	 Bias: -0.017917 	 Cost: 2.896727
Iter: 190 	 Weight: 0.327352 	 Bias: -0.018612 	 Cost: 2.895760
Iter: 195 	 Weight: 0.327369 	 Bias: -0.019307 	 Cost: 2.894794
Iter: 200 	 Weight: 0.327386 	 Bias: -0.020002 	 Cost: 2.893828
Iter: 205 	 Weight: 0.327403 	 Bias: -0.020696 	 Cost: 2.892862
Iter: 210 	 Weight: 0.327420 	 Bias: -0.021391 	 Cost: 2.891897
Iter: 215 	 Weight: 0.327437 	 Bias: -0.022085 	 Cost: 2.890932
Iter: 220 	 Weight: 0.327455 	 Bias: -0.022779 	 Cost: 2.889968
Iter: 225 	 Weight: 0.327472 	 Bias: -0.023473 	 Cost: 2.889005
Iter: 230 	 Weight: 0.327489 	 Bias: -0.024167 	 Cost: 2.888042
Iter: 235 	 Weight: 0.327506 	 Bias: -0.024860 	 Cost: 2.887079
Iter: 240 	 Weight: 0.327523 	 Bias: -0.025554 	 Cost: 2.886117
Iter: 245 	 Weight: 0.327540 	 Bias: -0.026247 	 Cost: 2.885155
Iter: 250 	 Weight: 0.327557 	 Bias: -0.026940 	 Cost: 2.884194
Iter: 255 	 Weight: 0.327574 	 Bias: -0.027633 	 Cost: 2.883233
Iter: 260 	 Weight: 0.327592 	 Bias: -0.028326 	 Cost: 2.882273
Iter: 265 	 Weight: 0.327609 	 Bias: -0.029018 	 Cost: 2.881313
Iter: 270 	 Weight: 0.327626 	 Bias: -0.029711 	 Cost: 2.880354
Iter: 275 	 Weight: 0.327643 	 Bias: -0.030403 	 Cost: 2.879395
Iter: 280 	 Weight: 0.327660 	 Bias: -0.031095 	 Cost: 2.878436
Iter: 285 	 Weight: 0.327677 	 Bias: -0.031787 	 Cost: 2.877479
Iter: 290 	 Weight: 0.327694 	 Bias: -0.032478 	 Cost: 2.876521
Iter: 295 	 Weight: 0.327711 	 Bias: -0.033170 	 Cost: 2.875564
Iter: 300 	 Weight: 0.327728 	 Bias: -0.033861 	 Cost: 2.874608
Iter: 305 	 Weight: 0.327745 	 Bias: -0.034552 	 Cost: 2.873652
Iter: 310 	 Weight: 0.327762 	 Bias: -0.035243 	 Cost: 2.872696
Iter: 315 	 Weight: 0.327779 	 Bias: -0.035934 	 Cost: 2.871741
Iter: 320 	 Weight: 0.327796 	 Bias: -0.036625 	 Cost: 2.870787
Iter: 325 	 Weight: 0.327813 	 Bias: -0.037315 	 Cost: 2.869832
Iter: 330 	 Weight: 0.327830 	 Bias: -0.038006 	 Cost: 2.868879
Iter: 335 	 Weight: 0.327848 	 Bias: -0.038696 	 Cost: 2.867926
Iter: 340 	 Weight: 0.327865 	 Bias: -0.039386 	 Cost: 2.866973
Iter: 345 	 Weight: 0.327882 	 Bias: -0.040076 	 Cost: 2.866021
Iter: 350 	 Weight: 0.327899 	 Bias: -0.040765 	 Cost: 2.865069
Iter: 355 	 Weight: 0.327916 	 Bias: -0.041455 	 Cost: 2.864118
Iter: 360 	 Weight: 0.327933 	 Bias: -0.042144 	 Cost: 2.863167
Iter: 365 	 Weight: 0.327950 	 Bias: -0.042833 	 Cost: 2.862217
Iter: 370 	 Weight: 0.327967 	 Bias: -0.043522 	 Cost: 2.861267
Iter: 375 	 Weight: 0.327984 	 Bias: -0.044211 	 Cost: 2.860317
Iter: 380 	 Weight: 0.328001 	 Bias: -0.044899 	 Cost: 2.859369
Iter: 385 	 Weight: 0.328018 	 Bias: -0.045588 	 Cost: 2.858420
Iter: 390 	 Weight: 0.328035 	 Bias: -0.046276 	 Cost: 2.857472
Iter: 395 	 Weight: 0.328052 	 Bias: -0.046964 	 Cost: 2.856525
Iter: 400 	 Weight: 0.328069 	 Bias: -0.047652 	 Cost: 2.855578
Iter: 405 	 Weight: 0.328086 	 Bias: -0.048340 	 Cost: 2.854631
Iter: 410 	 Weight: 0.328103 	 Bias: -0.049028 	 Cost: 2.853685
Iter: 415 	 Weight: 0.328120 	 Bias: -0.049715 	 Cost: 2.852739
Iter: 420 	 Weight: 0.328137 	 Bias: -0.050402 	 Cost: 2.851794
Iter: 425 	 Weight: 0.328153 	 Bias: -0.051089 	 Cost: 2.850850
Iter: 430 	 Weight: 0.328170 	 Bias: -0.051776 	 Cost: 2.849905
Iter: 435 	 Weight: 0.328187 	 Bias: -0.052463 	 Cost: 2.848962
Iter: 440 	 Weight: 0.328204 	 Bias: -0.053149 	 Cost: 2.848018
Iter: 445 	 Weight: 0.328221 	 Bias: -0.053836 	 Cost: 2.847076
Iter: 450 	 Weight: 0.328238 	 Bias: -0.054522 	 Cost: 2.846133
Iter: 455 	 Weight: 0.328255 	 Bias: -0.055208 	 Cost: 2.845191
Iter: 460 	 Weight: 0.328272 	 Bias: -0.055894 	 Cost: 2.844250
Iter: 465 	 Weight: 0.328289 	 Bias: -0.056580 	 Cost: 2.843309
Iter: 470 	 Weight: 0.328306 	 Bias: -0.057265 	 Cost: 2.842369
Iter: 475 	 Weight: 0.328323 	 Bias: -0.057951 	 Cost: 2.841429
Iter: 480 	 Weight: 0.328340 	 Bias: -0.058636 	 Cost: 2.840489
Iter: 485 	 Weight: 0.328357 	 Bias: -0.059321 	 Cost: 2.839550
Iter: 490 	 Weight: 0.328374 	 Bias: -0.060006 	 Cost: 2.838611
Iter: 495 	 Weight: 0.328390 	 Bias: -0.060690 	 Cost: 2.837673
Iter: 500 	 Weight: 0.328407 	 Bias: -0.061375 	 Cost: 2.836736
Iter: 505 	 Weight: 0.328424 	 Bias: -0.062059 	 Cost: 2.835798
Iter: 510 	 Weight: 0.328441 	 Bias: -0.062743 	 Cost: 2.834862
Iter: 515 	 Weight: 0.328458 	 Bias: -0.063427 	 Cost: 2.833925
Iter: 520 	 Weight: 0.328475 	 Bias: -0.064111 	 Cost: 2.832990
Iter: 525 	 Weight: 0.328492 	 Bias: -0.064795 	 Cost: 2.832054
Iter: 530 	 Weight: 0.328509 	 Bias: -0.065478 	 Cost: 2.831119
Iter: 535 	 Weight: 0.328526 	 Bias: -0.066162 	 Cost: 2.830185
Iter: 540 	 Weight: 0.328542 	 Bias: -0.066845 	 Cost: 2.829251
Iter: 545 	 Weight: 0.328559 	 Bias: -0.067528 	 Cost: 2.828318
Iter: 550 	 Weight: 0.328576 	 Bias: -0.068211 	 Cost: 2.827385
Iter: 555 	 Weight: 0.328593 	 Bias: -0.068893 	 Cost: 2.826452
Iter: 560 	 Weight: 0.328610 	 Bias: -0.069576 	 Cost: 2.825520
Iter: 565 	 Weight: 0.328627 	 Bias: -0.070258 	 Cost: 2.824588
Iter: 570 	 Weight: 0.328644 	 Bias: -0.070940 	 Cost: 2.823657
Iter: 575 	 Weight: 0.328660 	 Bias: -0.071622 	 Cost: 2.822727
Iter: 580 	 Weight: 0.328677 	 Bias: -0.072304 	 Cost: 2.821796
Iter: 585 	 Weight: 0.328694 	 Bias: -0.072985 	 Cost: 2.820867
Iter: 590 	 Weight: 0.328711 	 Bias: -0.073667 	 Cost: 2.819937
Iter: 595 	 Weight: 0.328728 	 Bias: -0.074348 	 Cost: 2.819008
Iter: 600 	 Weight: 0.328744 	 Bias: -0.075029 	 Cost: 2.818080
Iter: 605 	 Weight: 0.328761 	 Bias: -0.075710 	 Cost: 2.817152
Iter: 610 	 Weight: 0.328778 	 Bias: -0.076391 	 Cost: 2.816225
Iter: 615 	 Weight: 0.328795 	 Bias: -0.077072 	 Cost: 2.815298
Iter: 620 	 Weight: 0.328812 	 Bias: -0.077752 	 Cost: 2.814371
Iter: 625 	 Weight: 0.328828 	 Bias: -0.078432 	 Cost: 2.813445
Iter: 630 	 Weight: 0.328845 	 Bias: -0.079113 	 Cost: 2.812519
Iter: 635 	 Weight: 0.328862 	 Bias: -0.079792 	 Cost: 2.811594
Iter: 640 	 Weight: 0.328879 	 Bias: -0.080472 	 Cost: 2.810669
Iter: 645 	 Weight: 0.328896 	 Bias: -0.081152 	 Cost: 2.809745
Iter: 650 	 Weight: 0.328912 	 Bias: -0.081831 	 Cost: 2.808821
Iter: 655 	 Weight: 0.328929 	 Bias: -0.082511 	 Cost: 2.807898
Iter: 660 	 Weight: 0.328946 	 Bias: -0.083190 	 Cost: 2.806975
Iter: 665 	 Weight: 0.328963 	 Bias: -0.083869 	 Cost: 2.806053
Iter: 670 	 Weight: 0.328979 	 Bias: -0.084547 	 Cost: 2.805131
Iter: 675 	 Weight: 0.328996 	 Bias: -0.085226 	 Cost: 2.804209
Iter: 680 	 Weight: 0.329013 	 Bias: -0.085904 	 Cost: 2.803288
Iter: 685 	 Weight: 0.329030 	 Bias: -0.086583 	 Cost: 2.802368
Iter: 690 	 Weight: 0.329046 	 Bias: -0.087261 	 Cost: 2.801447
Iter: 695 	 Weight: 0.329063 	 Bias: -0.087939 	 Cost: 2.800528
Iter: 700 	 Weight: 0.329080 	 Bias: -0.088616 	 Cost: 2.799609
Iter: 705 	 Weight: 0.329097 	 Bias: -0.089294 	 Cost: 2.798690
Iter: 710 	 Weight: 0.329113 	 Bias: -0.089971 	 Cost: 2.797772
Iter: 715 	 Weight: 0.329130 	 Bias: -0.090649 	 Cost: 2.796854
Iter: 720 	 Weight: 0.329147 	 Bias: -0.091326 	 Cost: 2.795936
Iter: 725 	 Weight: 0.329163 	 Bias: -0.092003 	 Cost: 2.795019
Iter: 730 	 Weight: 0.329180 	 Bias: -0.092679 	 Cost: 2.794103
Iter: 735 	 Weight: 0.329197 	 Bias: -0.093356 	 Cost: 2.793187
Iter: 740 	 Weight: 0.329214 	 Bias: -0.094032 	 Cost: 2.792271
Iter: 745 	 Weight: 0.329230 	 Bias: -0.094708 	 Cost: 2.791356
Iter: 750 	 Weight: 0.329247 	 Bias: -0.095385 	 Cost: 2.790442
Iter: 755 	 Weight: 0.329264 	 Bias: -0.096060 	 Cost: 2.789527
Iter: 760 	 Weight: 0.329280 	 Bias: -0.096736 	 Cost: 2.788614
Iter: 765 	 Weight: 0.329297 	 Bias: -0.097412 	 Cost: 2.787700
Iter: 770 	 Weight: 0.329314 	 Bias: -0.098087 	 Cost: 2.786787
Iter: 775 	 Weight: 0.329330 	 Bias: -0.098762 	 Cost: 2.785875
Iter: 780 	 Weight: 0.329347 	 Bias: -0.099437 	 Cost: 2.784963
Iter: 785 	 Weight: 0.329364 	 Bias: -0.100112 	 Cost: 2.784052
Iter: 790 	 Weight: 0.329380 	 Bias: -0.100787 	 Cost: 2.783141
Iter: 795 	 Weight: 0.329397 	 Bias: -0.101462 	 Cost: 2.782230
Iter: 800 	 Weight: 0.329414 	 Bias: -0.102136 	 Cost: 2.781320
Iter: 805 	 Weight: 0.329430 	 Bias: -0.102810 	 Cost: 2.780410
Iter: 810 	 Weight: 0.329447 	 Bias: -0.103484 	 Cost: 2.779501
Iter: 815 	 Weight: 0.329464 	 Bias: -0.104158 	 Cost: 2.778592
Iter: 820 	 Weight: 0.329480 	 Bias: -0.104832 	 Cost: 2.777684
Iter: 825 	 Weight: 0.329497 	 Bias: -0.105505 	 Cost: 2.776776
Iter: 830 	 Weight: 0.329513 	 Bias: -0.106179 	 Cost: 2.775869
Iter: 835 	 Weight: 0.329530 	 Bias: -0.106852 	 Cost: 2.774962
Iter: 840 	 Weight: 0.329547 	 Bias: -0.107525 	 Cost: 2.774055
Iter: 845 	 Weight: 0.329563 	 Bias: -0.108198 	 Cost: 2.773149
Iter: 850 	 Weight: 0.329580 	 Bias: -0.108871 	 Cost: 2.772243
Iter: 855 	 Weight: 0.329597 	 Bias: -0.109543 	 Cost: 2.771338
Iter: 860 	 Weight: 0.329613 	 Bias: -0.110216 	 Cost: 2.770433
Iter: 865 	 Weight: 0.329630 	 Bias: -0.110888 	 Cost: 2.769529
Iter: 870 	 Weight: 0.329646 	 Bias: -0.111560 	 Cost: 2.768625
Iter: 875 	 Weight: 0.329663 	 Bias: -0.112232 	 Cost: 2.767722
Iter: 880 	 Weight: 0.329679 	 Bias: -0.112903 	 Cost: 2.766819
Iter: 885 	 Weight: 0.329696 	 Bias: -0.113575 	 Cost: 2.765917
Iter: 890 	 Weight: 0.329713 	 Bias: -0.114246 	 Cost: 2.765015
Iter: 895 	 Weight: 0.329729 	 Bias: -0.114918 	 Cost: 2.764113
Iter: 900 	 Weight: 0.329746 	 Bias: -0.115589 	 Cost: 2.763212
Iter: 905 	 Weight: 0.329762 	 Bias: -0.116259 	 Cost: 2.762311
Iter: 910 	 Weight: 0.329779 	 Bias: -0.116930 	 Cost: 2.761411
Iter: 915 	 Weight: 0.329795 	 Bias: -0.117601 	 Cost: 2.760511
Iter: 920 	 Weight: 0.329812 	 Bias: -0.118271 	 Cost: 2.759612
Iter: 925 	 Weight: 0.329829 	 Bias: -0.118941 	 Cost: 2.758713
Iter: 930 	 Weight: 0.329845 	 Bias: -0.119611 	 Cost: 2.757814
Iter: 935 	 Weight: 0.329862 	 Bias: -0.120281 	 Cost: 2.756916
Iter: 940 	 Weight: 0.329878 	 Bias: -0.120951 	 Cost: 2.756019
Iter: 945 	 Weight: 0.329895 	 Bias: -0.121621 	 Cost: 2.755122
Iter: 950 	 Weight: 0.329911 	 Bias: -0.122290 	 Cost: 2.754225
Iter: 955 	 Weight: 0.329928 	 Bias: -0.122959 	 Cost: 2.753329
Iter: 960 	 Weight: 0.329944 	 Bias: -0.123628 	 Cost: 2.752433
Iter: 965 	 Weight: 0.329961 	 Bias: -0.124297 	 Cost: 2.751538
Iter: 970 	 Weight: 0.329977 	 Bias: -0.124966 	 Cost: 2.750643
Iter: 975 	 Weight: 0.329994 	 Bias: -0.125634 	 Cost: 2.749748
Iter: 980 	 Weight: 0.330010 	 Bias: -0.126303 	 Cost: 2.748854
Iter: 985 	 Weight: 0.330027 	 Bias: -0.126971 	 Cost: 2.747961
Iter: 990 	 Weight: 0.330043 	 Bias: -0.127639 	 Cost: 2.747068
Iter: 995 	 Weight: 0.330060 	 Bias: -0.128307 	 Cost: 2.746175
Iter: 1000 	 Weight: 0.330076 	 Bias: -0.128975 	 Cost: 2.745283
Iter: 1005 	 Weight: 0.330093 	 Bias: -0.129642 	 Cost: 2.744391
Iter: 1010 	 Weight: 0.330109 	 Bias: -0.130310 	 Cost: 2.743500
Iter: 1015 	 Weight: 0.330126 	 Bias: -0.130977 	 Cost: 2.742609
Iter: 1020 	 Weight: 0.330142 	 Bias: -0.131644 	 Cost: 2.741718
Iter: 1025 	 Weight: 0.330159 	 Bias: -0.132311 	 Cost: 2.740828
Iter: 1030 	 Weight: 0.330175 	 Bias: -0.132977 	 Cost: 2.739939
Iter: 1035 	 Weight: 0.330191 	 Bias: -0.133644 	 Cost: 2.739050
Iter: 1040 	 Weight: 0.330208 	 Bias: -0.134310 	 Cost: 2.738161
Iter: 1045 	 Weight: 0.330224 	 Bias: -0.134977 	 Cost: 2.737273
Iter: 1050 	 Weight: 0.330241 	 Bias: -0.135643 	 Cost: 2.736385
Iter: 1055 	 Weight: 0.330257 	 Bias: -0.136309 	 Cost: 2.735497
Iter: 1060 	 Weight: 0.330274 	 Bias: -0.136974 	 Cost: 2.734610
Iter: 1065 	 Weight: 0.330290 	 Bias: -0.137640 	 Cost: 2.733724
Iter: 1070 	 Weight: 0.330307 	 Bias: -0.138305 	 Cost: 2.732838
Iter: 1075 	 Weight: 0.330323 	 Bias: -0.138971 	 Cost: 2.731952
Iter: 1080 	 Weight: 0.330339 	 Bias: -0.139636 	 Cost: 2.731067
Iter: 1085 	 Weight: 0.330356 	 Bias: -0.140301 	 Cost: 2.730182
Iter: 1090 	 Weight: 0.330372 	 Bias: -0.140965 	 Cost: 2.729298
Iter: 1095 	 Weight: 0.330389 	 Bias: -0.141630 	 Cost: 2.728414
Iter: 1100 	 Weight: 0.330405 	 Bias: -0.142294 	 Cost: 2.727531
Iter: 1105 	 Weight: 0.330421 	 Bias: -0.142959 	 Cost: 2.726648
Iter: 1110 	 Weight: 0.330438 	 Bias: -0.143623 	 Cost: 2.725765
Iter: 1115 	 Weight: 0.330454 	 Bias: -0.144287 	 Cost: 2.724883
Iter: 1120 	 Weight: 0.330471 	 Bias: -0.144950 	 Cost: 2.724002
Iter: 1125 	 Weight: 0.330487 	 Bias: -0.145614 	 Cost: 2.723120
Iter: 1130 	 Weight: 0.330503 	 Bias: -0.146277 	 Cost: 2.722240
Iter: 1135 	 Weight: 0.330520 	 Bias: -0.146941 	 Cost: 2.721359
Iter: 1140 	 Weight: 0.330536 	 Bias: -0.147604 	 Cost: 2.720479
Iter: 1145 	 Weight: 0.330552 	 Bias: -0.148267 	 Cost: 2.719600
Iter: 1150 	 Weight: 0.330569 	 Bias: -0.148929 	 Cost: 2.718721
Iter: 1155 	 Weight: 0.330585 	 Bias: -0.149592 	 Cost: 2.717842
Iter: 1160 	 Weight: 0.330602 	 Bias: -0.150254 	 Cost: 2.716964
Iter: 1165 	 Weight: 0.330618 	 Bias: -0.150917 	 Cost: 2.716086
Iter: 1170 	 Weight: 0.330634 	 Bias: -0.151579 	 Cost: 2.715209
Iter: 1175 	 Weight: 0.330651 	 Bias: -0.152241 	 Cost: 2.714332
Iter: 1180 	 Weight: 0.330667 	 Bias: -0.152903 	 Cost: 2.713456
Iter: 1185 	 Weight: 0.330683 	 Bias: -0.153564 	 Cost: 2.712580
Iter: 1190 	 Weight: 0.330700 	 Bias: -0.154226 	 Cost: 2.711704
Iter: 1195 	 Weight: 0.330716 	 Bias: -0.154887 	 Cost: 2.710829
Iter: 1200 	 Weight: 0.330732 	 Bias: -0.155548 	 Cost: 2.709955
Iter: 1205 	 Weight: 0.330749 	 Bias: -0.156209 	 Cost: 2.709080
Iter: 1210 	 Weight: 0.330765 	 Bias: -0.156870 	 Cost: 2.708206
Iter: 1215 	 Weight: 0.330781 	 Bias: -0.157530 	 Cost: 2.707333
Iter: 1220 	 Weight: 0.330797 	 Bias: -0.158191 	 Cost: 2.706460
Iter: 1225 	 Weight: 0.330814 	 Bias: -0.158851 	 Cost: 2.705588
Iter: 1230 	 Weight: 0.330830 	 Bias: -0.159511 	 Cost: 2.704716
Iter: 1235 	 Weight: 0.330846 	 Bias: -0.160171 	 Cost: 2.703844
Iter: 1240 	 Weight: 0.330863 	 Bias: -0.160831 	 Cost: 2.702973
Iter: 1245 	 Weight: 0.330879 	 Bias: -0.161491 	 Cost: 2.702102
Iter: 1250 	 Weight: 0.330895 	 Bias: -0.162150 	 Cost: 2.701232
Iter: 1255 	 Weight: 0.330911 	 Bias: -0.162810 	 Cost: 2.700362
Iter: 1260 	 Weight: 0.330928 	 Bias: -0.163469 	 Cost: 2.699492
Iter: 1265 	 Weight: 0.330944 	 Bias: -0.164128 	 Cost: 2.698623
Iter: 1270 	 Weight: 0.330960 	 Bias: -0.164787 	 Cost: 2.697755
Iter: 1275 	 Weight: 0.330977 	 Bias: -0.165445 	 Cost: 2.696886
Iter: 1280 	 Weight: 0.330993 	 Bias: -0.166104 	 Cost: 2.696019
Iter: 1285 	 Weight: 0.331009 	 Bias: -0.166762 	 Cost: 2.695151
Iter: 1290 	 Weight: 0.331025 	 Bias: -0.167420 	 Cost: 2.694284
Iter: 1295 	 Weight: 0.331042 	 Bias: -0.168078 	 Cost: 2.693418
Iter: 1300 	 Weight: 0.331058 	 Bias: -0.168736 	 Cost: 2.692552
Iter: 1305 	 Weight: 0.331074 	 Bias: -0.169394 	 Cost: 2.691686
Iter: 1310 	 Weight: 0.331090 	 Bias: -0.170051 	 Cost: 2.690821
Iter: 1315 	 Weight: 0.331107 	 Bias: -0.170709 	 Cost: 2.689956
Iter: 1320 	 Weight: 0.331123 	 Bias: -0.171366 	 Cost: 2.689092
Iter: 1325 	 Weight: 0.331139 	 Bias: -0.172023 	 Cost: 2.688228
Iter: 1330 	 Weight: 0.331155 	 Bias: -0.172680 	 Cost: 2.687365
Iter: 1335 	 Weight: 0.331171 	 Bias: -0.173336 	 Cost: 2.686502
Iter: 1340 	 Weight: 0.331188 	 Bias: -0.173993 	 Cost: 2.685639
Iter: 1345 	 Weight: 0.331204 	 Bias: -0.174649 	 Cost: 2.684777
Iter: 1350 	 Weight: 0.331220 	 Bias: -0.175306 	 Cost: 2.683915
Iter: 1355 	 Weight: 0.331236 	 Bias: -0.175962 	 Cost: 2.683054
Iter: 1360 	 Weight: 0.331252 	 Bias: -0.176618 	 Cost: 2.682193
Iter: 1365 	 Weight: 0.331269 	 Bias: -0.177273 	 Cost: 2.681332
Iter: 1370 	 Weight: 0.331285 	 Bias: -0.177929 	 Cost: 2.680472
Iter: 1375 	 Weight: 0.331301 	 Bias: -0.178584 	 Cost: 2.679613
Iter: 1380 	 Weight: 0.331317 	 Bias: -0.179239 	 Cost: 2.678754
Iter: 1385 	 Weight: 0.331333 	 Bias: -0.179895 	 Cost: 2.677895
Iter: 1390 	 Weight: 0.331349 	 Bias: -0.180549 	 Cost: 2.677037
Iter: 1395 	 Weight: 0.331366 	 Bias: -0.181204 	 Cost: 2.676179
Iter: 1400 	 Weight: 0.331382 	 Bias: -0.181859 	 Cost: 2.675321
Iter: 1405 	 Weight: 0.331398 	 Bias: -0.182513 	 Cost: 2.674464
Iter: 1410 	 Weight: 0.331414 	 Bias: -0.183167 	 Cost: 2.673607
Iter: 1415 	 Weight: 0.331430 	 Bias: -0.183822 	 Cost: 2.672751
Iter: 1420 	 Weight: 0.331446 	 Bias: -0.184476 	 Cost: 2.671896
Iter: 1425 	 Weight: 0.331463 	 Bias: -0.185129 	 Cost: 2.671040
Iter: 1430 	 Weight: 0.331479 	 Bias: -0.185783 	 Cost: 2.670185
Iter: 1435 	 Weight: 0.331495 	 Bias: -0.186436 	 Cost: 2.669331
Iter: 1440 	 Weight: 0.331511 	 Bias: -0.187090 	 Cost: 2.668477
Iter: 1445 	 Weight: 0.331527 	 Bias: -0.187743 	 Cost: 2.667623
Iter: 1450 	 Weight: 0.331543 	 Bias: -0.188396 	 Cost: 2.666770
Iter: 1455 	 Weight: 0.331559 	 Bias: -0.189049 	 Cost: 2.665917
Iter: 1460 	 Weight: 0.331575 	 Bias: -0.189701 	 Cost: 2.665065
Iter: 1465 	 Weight: 0.331591 	 Bias: -0.190354 	 Cost: 2.664213
Iter: 1470 	 Weight: 0.331608 	 Bias: -0.191006 	 Cost: 2.663361
Iter: 1475 	 Weight: 0.331624 	 Bias: -0.191658 	 Cost: 2.662510
Iter: 1480 	 Weight: 0.331640 	 Bias: -0.192310 	 Cost: 2.661659
Iter: 1485 	 Weight: 0.331656 	 Bias: -0.192962 	 Cost: 2.660809
Iter: 1490 	 Weight: 0.331672 	 Bias: -0.193614 	 Cost: 2.659959
Iter: 1495 	 Weight: 0.331688 	 Bias: -0.194265 	 Cost: 2.659110
Iter: 1500 	 Weight: 0.331704 	 Bias: -0.194917 	 Cost: 2.658261
Iter: 1505 	 Weight: 0.331720 	 Bias: -0.195568 	 Cost: 2.657412
Iter: 1510 	 Weight: 0.331736 	 Bias: -0.196219 	 Cost: 2.656564
Iter: 1515 	 Weight: 0.331752 	 Bias: -0.196870 	 Cost: 2.655716
Iter: 1520 	 Weight: 0.331768 	 Bias: -0.197520 	 Cost: 2.654869
Iter: 1525 	 Weight: 0.331784 	 Bias: -0.198171 	 Cost: 2.654022
Iter: 1530 	 Weight: 0.331801 	 Bias: -0.198821 	 Cost: 2.653176
Iter: 1535 	 Weight: 0.331817 	 Bias: -0.199471 	 Cost: 2.652330
Iter: 1540 	 Weight: 0.331833 	 Bias: -0.200121 	 Cost: 2.651484
Iter: 1545 	 Weight: 0.331849 	 Bias: -0.200771 	 Cost: 2.650639
Iter: 1550 	 Weight: 0.331865 	 Bias: -0.201421 	 Cost: 2.649794
Iter: 1555 	 Weight: 0.331881 	 Bias: -0.202071 	 Cost: 2.648950
Iter: 1560 	 Weight: 0.331897 	 Bias: -0.202720 	 Cost: 2.648106
Iter: 1565 	 Weight: 0.331913 	 Bias: -0.203369 	 Cost: 2.647262
Iter: 1570 	 Weight: 0.331929 	 Bias: -0.204018 	 Cost: 2.646419
Iter: 1575 	 Weight: 0.331945 	 Bias: -0.204667 	 Cost: 2.645576
Iter: 1580 	 Weight: 0.331961 	 Bias: -0.205316 	 Cost: 2.644734
Iter: 1585 	 Weight: 0.331977 	 Bias: -0.205965 	 Cost: 2.643892
Iter: 1590 	 Weight: 0.331993 	 Bias: -0.206613 	 Cost: 2.643051
Iter: 1595 	 Weight: 0.332009 	 Bias: -0.207261 	 Cost: 2.642210
Iter: 1600 	 Weight: 0.332025 	 Bias: -0.207910 	 Cost: 2.641369
Iter: 1605 	 Weight: 0.332041 	 Bias: -0.208557 	 Cost: 2.640529
Iter: 1610 	 Weight: 0.332057 	 Bias: -0.209205 	 Cost: 2.639689
Iter: 1615 	 Weight: 0.332073 	 Bias: -0.209853 	 Cost: 2.638850
Iter: 1620 	 Weight: 0.332089 	 Bias: -0.210500 	 Cost: 2.638011
Iter: 1625 	 Weight: 0.332105 	 Bias: -0.211148 	 Cost: 2.637172
Iter: 1630 	 Weight: 0.332121 	 Bias: -0.211795 	 Cost: 2.636334
Iter: 1635 	 Weight: 0.332137 	 Bias: -0.212442 	 Cost: 2.635496
Iter: 1640 	 Weight: 0.332153 	 Bias: -0.213089 	 Cost: 2.634659
Iter: 1645 	 Weight: 0.332169 	 Bias: -0.213735 	 Cost: 2.633822
Iter: 1650 	 Weight: 0.332185 	 Bias: -0.214382 	 Cost: 2.632986
Iter: 1655 	 Weight: 0.332201 	 Bias: -0.215028 	 Cost: 2.632150
Iter: 1660 	 Weight: 0.332217 	 Bias: -0.215674 	 Cost: 2.631314
Iter: 1665 	 Weight: 0.332233 	 Bias: -0.216320 	 Cost: 2.630479
Iter: 1670 	 Weight: 0.332248 	 Bias: -0.216966 	 Cost: 2.629644
Iter: 1675 	 Weight: 0.332264 	 Bias: -0.217612 	 Cost: 2.628810
Iter: 1680 	 Weight: 0.332280 	 Bias: -0.218258 	 Cost: 2.627976
Iter: 1685 	 Weight: 0.332296 	 Bias: -0.218903 	 Cost: 2.627142
Iter: 1690 	 Weight: 0.332312 	 Bias: -0.219548 	 Cost: 2.626309
Iter: 1695 	 Weight: 0.332328 	 Bias: -0.220193 	 Cost: 2.625477
Iter: 1700 	 Weight: 0.332344 	 Bias: -0.220838 	 Cost: 2.624644
Iter: 1705 	 Weight: 0.332360 	 Bias: -0.221483 	 Cost: 2.623812
Iter: 1710 	 Weight: 0.332376 	 Bias: -0.222128 	 Cost: 2.622981
Iter: 1715 	 Weight: 0.332392 	 Bias: -0.222772 	 Cost: 2.622150
Iter: 1720 	 Weight: 0.332408 	 Bias: -0.223416 	 Cost: 2.621319
Iter: 1725 	 Weight: 0.332424 	 Bias: -0.224060 	 Cost: 2.620489
Iter: 1730 	 Weight: 0.332439 	 Bias: -0.224704 	 Cost: 2.619659
Iter: 1735 	 Weight: 0.332455 	 Bias: -0.225348 	 Cost: 2.618830
Iter: 1740 	 Weight: 0.332471 	 Bias: -0.225992 	 Cost: 2.618001
Iter: 1745 	 Weight: 0.332487 	 Bias: -0.226635 	 Cost: 2.617172
Iter: 1750 	 Weight: 0.332503 	 Bias: -0.227278 	 Cost: 2.616344
Iter: 1755 	 Weight: 0.332519 	 Bias: -0.227922 	 Cost: 2.615516
Iter: 1760 	 Weight: 0.332535 	 Bias: -0.228565 	 Cost: 2.614689
Iter: 1765 	 Weight: 0.332551 	 Bias: -0.229207 	 Cost: 2.613862
Iter: 1770 	 Weight: 0.332567 	 Bias: -0.229850 	 Cost: 2.613035
Iter: 1775 	 Weight: 0.332582 	 Bias: -0.230493 	 Cost: 2.612209
Iter: 1780 	 Weight: 0.332598 	 Bias: -0.231135 	 Cost: 2.611383
Iter: 1785 	 Weight: 0.332614 	 Bias: -0.231777 	 Cost: 2.610558
Iter: 1790 	 Weight: 0.332630 	 Bias: -0.232419 	 Cost: 2.609733
Iter: 1795 	 Weight: 0.332646 	 Bias: -0.233061 	 Cost: 2.608909
Iter: 1800 	 Weight: 0.332662 	 Bias: -0.233703 	 Cost: 2.608085
Iter: 1805 	 Weight: 0.332677 	 Bias: -0.234344 	 Cost: 2.607261
Iter: 1810 	 Weight: 0.332693 	 Bias: -0.234986 	 Cost: 2.606438
Iter: 1815 	 Weight: 0.332709 	 Bias: -0.235627 	 Cost: 2.605615
Iter: 1820 	 Weight: 0.332725 	 Bias: -0.236268 	 Cost: 2.604793
Iter: 1825 	 Weight: 0.332741 	 Bias: -0.236909 	 Cost: 2.603970
Iter: 1830 	 Weight: 0.332757 	 Bias: -0.237550 	 Cost: 2.603149
Iter: 1835 	 Weight: 0.332772 	 Bias: -0.238190 	 Cost: 2.602328
Iter: 1840 	 Weight: 0.332788 	 Bias: -0.238831 	 Cost: 2.601507
Iter: 1845 	 Weight: 0.332804 	 Bias: -0.239471 	 Cost: 2.600686
Iter: 1850 	 Weight: 0.332820 	 Bias: -0.240111 	 Cost: 2.599866
Iter: 1855 	 Weight: 0.332836 	 Bias: -0.240751 	 Cost: 2.599047
Iter: 1860 	 Weight: 0.332851 	 Bias: -0.241391 	 Cost: 2.598228
Iter: 1865 	 Weight: 0.332867 	 Bias: -0.242031 	 Cost: 2.597409
Iter: 1870 	 Weight: 0.332883 	 Bias: -0.242670 	 Cost: 2.596591
Iter: 1875 	 Weight: 0.332899 	 Bias: -0.243309 	 Cost: 2.595773
Iter: 1880 	 Weight: 0.332915 	 Bias: -0.243949 	 Cost: 2.594955
Iter: 1885 	 Weight: 0.332930 	 Bias: -0.244588 	 Cost: 2.594138
Iter: 1890 	 Weight: 0.332946 	 Bias: -0.245226 	 Cost: 2.593321
Iter: 1895 	 Weight: 0.332962 	 Bias: -0.245865 	 Cost: 2.592505
Iter: 1900 	 Weight: 0.332978 	 Bias: -0.246504 	 Cost: 2.591689
Iter: 1905 	 Weight: 0.332993 	 Bias: -0.247142 	 Cost: 2.590873
Iter: 1910 	 Weight: 0.333009 	 Bias: -0.247780 	 Cost: 2.590058
Iter: 1915 	 Weight: 0.333025 	 Bias: -0.248418 	 Cost: 2.589244
Iter: 1920 	 Weight: 0.333041 	 Bias: -0.249056 	 Cost: 2.588429
Iter: 1925 	 Weight: 0.333056 	 Bias: -0.249694 	 Cost: 2.587615
Iter: 1930 	 Weight: 0.333072 	 Bias: -0.250332 	 Cost: 2.586802
Iter: 1935 	 Weight: 0.333088 	 Bias: -0.250969 	 Cost: 2.585989
Iter: 1940 	 Weight: 0.333104 	 Bias: -0.251606 	 Cost: 2.585176
Iter: 1945 	 Weight: 0.333119 	 Bias: -0.252243 	 Cost: 2.584364
Iter: 1950 	 Weight: 0.333135 	 Bias: -0.252880 	 Cost: 2.583552
Iter: 1955 	 Weight: 0.333151 	 Bias: -0.253517 	 Cost: 2.582740
Iter: 1960 	 Weight: 0.333167 	 Bias: -0.254154 	 Cost: 2.581929
Iter: 1965 	 Weight: 0.333182 	 Bias: -0.254790 	 Cost: 2.581119
Iter: 1970 	 Weight: 0.333198 	 Bias: -0.255426 	 Cost: 2.580308
Iter: 1975 	 Weight: 0.333214 	 Bias: -0.256063 	 Cost: 2.579498
Iter: 1980 	 Weight: 0.333229 	 Bias: -0.256699 	 Cost: 2.578689
Iter: 1985 	 Weight: 0.333245 	 Bias: -0.257335 	 Cost: 2.577880
Iter: 1990 	 Weight: 0.333261 	 Bias: -0.257970 	 Cost: 2.577071
Iter: 1995 	 Weight: 0.333276 	 Bias: -0.258606 	 Cost: 2.576263

iters = np.arange(2000)
plt.plot(iters, cost_history)

[]

We can see, based on the initial weight=0,bias=0, learning_rate=0.0001,the cost value decrease sharply.

sales_predict = weight_final*radio_sample + bias_final
plt.scatter(radio_sample, sales_sample, alpha=0.5)
plt.plot(radio_sample, sales_predict, c="r")
plt.xlabel("radio")
plt.ylabel("sales")

Text(0, 0.5, 'sales')

Experiment 2

In the last experiment, we set four hyperparameters manually which are w, b, l and i. Let’s change them and see what happen.

weight_final, bias_final, cost_history = train(radio_sample, sales_sample, -1000, 0, 0.0001, 2000)

Iter: 0 	 Weight: -753.190221 	 Bias: 6.091610 	 Cost: 700169768.718390
Iter: 5 	 Weight: -182.698756 	 Bias: 20.168463 	 Cost: 41100789.097582
Iter: 10 	 Weight: -44.478396 	 Bias: 23.573870 	 Cost: 2412831.778656
Iter: 15 	 Weight: -10.989863 	 Bias: 24.393771 	 Cost: 141812.898689
Iter: 20 	 Weight: -2.876041 	 Bias: 24.587251 	 Cost: 8501.907289
Iter: 25 	 Weight: -0.910075 	 Bias: 24.628960 	 Cost: 676.341469
Iter: 30 	 Weight: -0.433628 	 Bias: 24.633899 	 Cost: 216.885960
Iter: 35 	 Weight: -0.318065 	 Bias: 24.629931 	 Cost: 189.827967
Iter: 40 	 Weight: -0.289939 	 Bias: 24.623806 	 Cost: 188.152151
Iter: 45 	 Weight: -0.282997 	 Bias: 24.617160 	 Cost: 187.966335
Iter: 50 	 Weight: -0.281188 	 Bias: 24.610389 	 Cost: 187.868027
Iter: 55 	 Weight: -0.280622 	 Bias: 24.603589 	 Cost: 187.774899
Iter: 60 	 Weight: -0.280358 	 Bias: 24.596783 	 Cost: 187.682118
Iter: 65 	 Weight: -0.280166 	 Bias: 24.589976 	 Cost: 187.589402
Iter: 70 	 Weight: -0.279993 	 Bias: 24.583172 	 Cost: 187.496732
Iter: 75 	 Weight: -0.279823 	 Bias: 24.576368 	 Cost: 187.404109
Iter: 80 	 Weight: -0.279655 	 Bias: 24.569567 	 Cost: 187.311531
Iter: 85 	 Weight: -0.279487 	 Bias: 24.562767 	 Cost: 187.218999
Iter: 90 	 Weight: -0.279319 	 Bias: 24.555969 	 Cost: 187.126514
Iter: 95 	 Weight: -0.279152 	 Bias: 24.549172 	 Cost: 187.034074
Iter: 100 	 Weight: -0.278984 	 Bias: 24.542377 	 Cost: 186.941680
Iter: 105 	 Weight: -0.278816 	 Bias: 24.535584 	 Cost: 186.849332
Iter: 110 	 Weight: -0.278648 	 Bias: 24.528793 	 Cost: 186.757030
Iter: 115 	 Weight: -0.278481 	 Bias: 24.522003 	 Cost: 186.664774
Iter: 120 	 Weight: -0.278313 	 Bias: 24.515215 	 Cost: 186.572564
Iter: 125 	 Weight: -0.278146 	 Bias: 24.508428 	 Cost: 186.480399
Iter: 130 	 Weight: -0.277978 	 Bias: 24.501644 	 Cost: 186.388280
Iter: 135 	 Weight: -0.277811 	 Bias: 24.494861 	 Cost: 186.296207
Iter: 140 	 Weight: -0.277643 	 Bias: 24.488079 	 Cost: 186.204179
Iter: 145 	 Weight: -0.277476 	 Bias: 24.481300 	 Cost: 186.112198
Iter: 150 	 Weight: -0.277309 	 Bias: 24.474522 	 Cost: 186.020261
Iter: 155 	 Weight: -0.277141 	 Bias: 24.467745 	 Cost: 185.928371
Iter: 160 	 Weight: -0.276974 	 Bias: 24.460971 	 Cost: 185.836526
Iter: 165 	 Weight: -0.276807 	 Bias: 24.454198 	 Cost: 185.744727
Iter: 170 	 Weight: -0.276640 	 Bias: 24.447426 	 Cost: 185.652973
Iter: 175 	 Weight: -0.276473 	 Bias: 24.440657 	 Cost: 185.561265
Iter: 180 	 Weight: -0.276306 	 Bias: 24.433889 	 Cost: 185.469603
Iter: 185 	 Weight: -0.276138 	 Bias: 24.427123 	 Cost: 185.377986
Iter: 190 	 Weight: -0.275971 	 Bias: 24.420358 	 Cost: 185.286414
Iter: 195 	 Weight: -0.275805 	 Bias: 24.413595 	 Cost: 185.194888
Iter: 200 	 Weight: -0.275638 	 Bias: 24.406834 	 Cost: 185.103407
Iter: 205 	 Weight: -0.275471 	 Bias: 24.400075 	 Cost: 185.011972
Iter: 210 	 Weight: -0.275304 	 Bias: 24.393317 	 Cost: 184.920582
Iter: 215 	 Weight: -0.275137 	 Bias: 24.386561 	 Cost: 184.829238
Iter: 220 	 Weight: -0.274970 	 Bias: 24.379806 	 Cost: 184.737939
Iter: 225 	 Weight: -0.274804 	 Bias: 24.373053 	 Cost: 184.646685
Iter: 230 	 Weight: -0.274637 	 Bias: 24.366302 	 Cost: 184.555477
Iter: 235 	 Weight: -0.274470 	 Bias: 24.359553 	 Cost: 184.464314
Iter: 240 	 Weight: -0.274304 	 Bias: 24.352805 	 Cost: 184.373196
Iter: 245 	 Weight: -0.274137 	 Bias: 24.346059 	 Cost: 184.282123
Iter: 250 	 Weight: -0.273971 	 Bias: 24.339315 	 Cost: 184.191096
Iter: 255 	 Weight: -0.273804 	 Bias: 24.332572 	 Cost: 184.100113
Iter: 260 	 Weight: -0.273638 	 Bias: 24.325831 	 Cost: 184.009176
Iter: 265 	 Weight: -0.273472 	 Bias: 24.319091 	 Cost: 183.918284
Iter: 270 	 Weight: -0.273305 	 Bias: 24.312354 	 Cost: 183.827438
Iter: 275 	 Weight: -0.273139 	 Bias: 24.305618 	 Cost: 183.736636
Iter: 280 	 Weight: -0.272973 	 Bias: 24.298883 	 Cost: 183.645879
Iter: 285 	 Weight: -0.272806 	 Bias: 24.292150 	 Cost: 183.555168
Iter: 290 	 Weight: -0.272640 	 Bias: 24.285419 	 Cost: 183.464501
Iter: 295 	 Weight: -0.272474 	 Bias: 24.278690 	 Cost: 183.373880
Iter: 300 	 Weight: -0.272308 	 Bias: 24.271962 	 Cost: 183.283303
Iter: 305 	 Weight: -0.272142 	 Bias: 24.265236 	 Cost: 183.192772
Iter: 310 	 Weight: -0.271976 	 Bias: 24.258512 	 Cost: 183.102285
Iter: 315 	 Weight: -0.271810 	 Bias: 24.251789 	 Cost: 183.011844
Iter: 320 	 Weight: -0.271644 	 Bias: 24.245068 	 Cost: 182.921447
Iter: 325 	 Weight: -0.271478 	 Bias: 24.238349 	 Cost: 182.831095
Iter: 330 	 Weight: -0.271312 	 Bias: 24.231631 	 Cost: 182.740788
Iter: 335 	 Weight: -0.271147 	 Bias: 24.224915 	 Cost: 182.650526
Iter: 340 	 Weight: -0.270981 	 Bias: 24.218201 	 Cost: 182.560309
Iter: 345 	 Weight: -0.270815 	 Bias: 24.211488 	 Cost: 182.470136
Iter: 350 	 Weight: -0.270649 	 Bias: 24.204777 	 Cost: 182.380008
Iter: 355 	 Weight: -0.270484 	 Bias: 24.198068 	 Cost: 182.289925
Iter: 360 	 Weight: -0.270318 	 Bias: 24.191360 	 Cost: 182.199887
Iter: 365 	 Weight: -0.270153 	 Bias: 24.184654 	 Cost: 182.109893
Iter: 370 	 Weight: -0.269987 	 Bias: 24.177950 	 Cost: 182.019945
Iter: 375 	 Weight: -0.269822 	 Bias: 24.171247 	 Cost: 181.930040
Iter: 380 	 Weight: -0.269656 	 Bias: 24.164546 	 Cost: 181.840181
Iter: 385 	 Weight: -0.269491 	 Bias: 24.157847 	 Cost: 181.750366
Iter: 390 	 Weight: -0.269326 	 Bias: 24.151149 	 Cost: 181.660595
Iter: 395 	 Weight: -0.269160 	 Bias: 24.144453 	 Cost: 181.570870
Iter: 400 	 Weight: -0.268995 	 Bias: 24.137759 	 Cost: 181.481188
Iter: 405 	 Weight: -0.268830 	 Bias: 24.131066 	 Cost: 181.391552
Iter: 410 	 Weight: -0.268665 	 Bias: 24.124375 	 Cost: 181.301959
Iter: 415 	 Weight: -0.268499 	 Bias: 24.117686 	 Cost: 181.212412
Iter: 420 	 Weight: -0.268334 	 Bias: 24.110998 	 Cost: 181.122908
Iter: 425 	 Weight: -0.268169 	 Bias: 24.104312 	 Cost: 181.033450
Iter: 430 	 Weight: -0.268004 	 Bias: 24.097628 	 Cost: 180.944035
Iter: 435 	 Weight: -0.267839 	 Bias: 24.090945 	 Cost: 180.854665
Iter: 440 	 Weight: -0.267674 	 Bias: 24.084264 	 Cost: 180.765339
Iter: 445 	 Weight: -0.267509 	 Bias: 24.077584 	 Cost: 180.676058
Iter: 450 	 Weight: -0.267345 	 Bias: 24.070907 	 Cost: 180.586821
Iter: 455 	 Weight: -0.267180 	 Bias: 24.064231 	 Cost: 180.497629
Iter: 460 	 Weight: -0.267015 	 Bias: 24.057556 	 Cost: 180.408480
Iter: 465 	 Weight: -0.266850 	 Bias: 24.050883 	 Cost: 180.319376
Iter: 470 	 Weight: -0.266686 	 Bias: 24.044212 	 Cost: 180.230316
Iter: 475 	 Weight: -0.266521 	 Bias: 24.037543 	 Cost: 180.141300
Iter: 480 	 Weight: -0.266356 	 Bias: 24.030875 	 Cost: 180.052329
Iter: 485 	 Weight: -0.266192 	 Bias: 24.024209 	 Cost: 179.963402
Iter: 490 	 Weight: -0.266027 	 Bias: 24.017544 	 Cost: 179.874519
Iter: 495 	 Weight: -0.265863 	 Bias: 24.010881 	 Cost: 179.785680
Iter: 500 	 Weight: -0.265698 	 Bias: 24.004220 	 Cost: 179.696885
Iter: 505 	 Weight: -0.265534 	 Bias: 23.997561 	 Cost: 179.608134
Iter: 510 	 Weight: -0.265370 	 Bias: 23.990903 	 Cost: 179.519427
Iter: 515 	 Weight: -0.265205 	 Bias: 23.984247 	 Cost: 179.430765
Iter: 520 	 Weight: -0.265041 	 Bias: 23.977592 	 Cost: 179.342146
Iter: 525 	 Weight: -0.264877 	 Bias: 23.970939 	 Cost: 179.253571
Iter: 530 	 Weight: -0.264713 	 Bias: 23.964288 	 Cost: 179.165041
Iter: 535 	 Weight: -0.264548 	 Bias: 23.957638 	 Cost: 179.076554
Iter: 540 	 Weight: -0.264384 	 Bias: 23.950990 	 Cost: 178.988111
Iter: 545 	 Weight: -0.264220 	 Bias: 23.944344 	 Cost: 178.899712
Iter: 550 	 Weight: -0.264056 	 Bias: 23.937699 	 Cost: 178.811357
Iter: 555 	 Weight: -0.263892 	 Bias: 23.931056 	 Cost: 178.723046
Iter: 560 	 Weight: -0.263728 	 Bias: 23.924415 	 Cost: 178.634779
Iter: 565 	 Weight: -0.263564 	 Bias: 23.917775 	 Cost: 178.546555
Iter: 570 	 Weight: -0.263400 	 Bias: 23.911137 	 Cost: 178.458376
Iter: 575 	 Weight: -0.263237 	 Bias: 23.904501 	 Cost: 178.370240
Iter: 580 	 Weight: -0.263073 	 Bias: 23.897866 	 Cost: 178.282148
Iter: 585 	 Weight: -0.262909 	 Bias: 23.891233 	 Cost: 178.194099
Iter: 590 	 Weight: -0.262745 	 Bias: 23.884601 	 Cost: 178.106095
Iter: 595 	 Weight: -0.262582 	 Bias: 23.877971 	 Cost: 178.018134
Iter: 600 	 Weight: -0.262418 	 Bias: 23.871343 	 Cost: 177.930216
Iter: 605 	 Weight: -0.262254 	 Bias: 23.864717 	 Cost: 177.842343
Iter: 610 	 Weight: -0.262091 	 Bias: 23.858092 	 Cost: 177.754513
Iter: 615 	 Weight: -0.261927 	 Bias: 23.851468 	 Cost: 177.666726
Iter: 620 	 Weight: -0.261764 	 Bias: 23.844847 	 Cost: 177.578984
Iter: 625 	 Weight: -0.261600 	 Bias: 23.838227 	 Cost: 177.491284
Iter: 630 	 Weight: -0.261437 	 Bias: 23.831609 	 Cost: 177.403629
Iter: 635 	 Weight: -0.261274 	 Bias: 23.824992 	 Cost: 177.316017
Iter: 640 	 Weight: -0.261110 	 Bias: 23.818377 	 Cost: 177.228448
Iter: 645 	 Weight: -0.260947 	 Bias: 23.811763 	 Cost: 177.140923
Iter: 650 	 Weight: -0.260784 	 Bias: 23.805152 	 Cost: 177.053441
Iter: 655 	 Weight: -0.260621 	 Bias: 23.798542 	 Cost: 176.966003
Iter: 660 	 Weight: -0.260458 	 Bias: 23.791933 	 Cost: 176.878608
Iter: 665 	 Weight: -0.260295 	 Bias: 23.785326 	 Cost: 176.791256
Iter: 670 	 Weight: -0.260131 	 Bias: 23.778721 	 Cost: 176.703948
Iter: 675 	 Weight: -0.259968 	 Bias: 23.772117 	 Cost: 176.616683
Iter: 680 	 Weight: -0.259805 	 Bias: 23.765516 	 Cost: 176.529462
Iter: 685 	 Weight: -0.259642 	 Bias: 23.758915 	 Cost: 176.442284
Iter: 690 	 Weight: -0.259480 	 Bias: 23.752317 	 Cost: 176.355149
Iter: 695 	 Weight: -0.259317 	 Bias: 23.745720 	 Cost: 176.268058
Iter: 700 	 Weight: -0.259154 	 Bias: 23.739124 	 Cost: 176.181009
Iter: 705 	 Weight: -0.258991 	 Bias: 23.732531 	 Cost: 176.094004
Iter: 710 	 Weight: -0.258828 	 Bias: 23.725939 	 Cost: 176.007042
Iter: 715 	 Weight: -0.258666 	 Bias: 23.719348 	 Cost: 175.920123
Iter: 720 	 Weight: -0.258503 	 Bias: 23.712759 	 Cost: 175.833248
Iter: 725 	 Weight: -0.258340 	 Bias: 23.706172 	 Cost: 175.746415
Iter: 730 	 Weight: -0.258178 	 Bias: 23.699587 	 Cost: 175.659626
Iter: 735 	 Weight: -0.258015 	 Bias: 23.693003 	 Cost: 175.572880
Iter: 740 	 Weight: -0.257853 	 Bias: 23.686420 	 Cost: 175.486177
Iter: 745 	 Weight: -0.257690 	 Bias: 23.679840 	 Cost: 175.399517
Iter: 750 	 Weight: -0.257528 	 Bias: 23.673261 	 Cost: 175.312899
Iter: 755 	 Weight: -0.257366 	 Bias: 23.666683 	 Cost: 175.226325
Iter: 760 	 Weight: -0.257203 	 Bias: 23.660108 	 Cost: 175.139794
Iter: 765 	 Weight: -0.257041 	 Bias: 23.653534 	 Cost: 175.053306
Iter: 770 	 Weight: -0.256879 	 Bias: 23.646961 	 Cost: 174.966861
Iter: 775 	 Weight: -0.256716 	 Bias: 23.640390 	 Cost: 174.880459
Iter: 780 	 Weight: -0.256554 	 Bias: 23.633821 	 Cost: 174.794099
Iter: 785 	 Weight: -0.256392 	 Bias: 23.627253 	 Cost: 174.707783
Iter: 790 	 Weight: -0.256230 	 Bias: 23.620688 	 Cost: 174.621509
Iter: 795 	 Weight: -0.256068 	 Bias: 23.614123 	 Cost: 174.535278
Iter: 800 	 Weight: -0.255906 	 Bias: 23.607561 	 Cost: 174.449090
Iter: 805 	 Weight: -0.255744 	 Bias: 23.600999 	 Cost: 174.362945
Iter: 810 	 Weight: -0.255582 	 Bias: 23.594440 	 Cost: 174.276843
Iter: 815 	 Weight: -0.255420 	 Bias: 23.587882 	 Cost: 174.190783
Iter: 820 	 Weight: -0.255258 	 Bias: 23.581326 	 Cost: 174.104766
Iter: 825 	 Weight: -0.255097 	 Bias: 23.574772 	 Cost: 174.018792
Iter: 830 	 Weight: -0.254935 	 Bias: 23.568219 	 Cost: 173.932860
Iter: 835 	 Weight: -0.254773 	 Bias: 23.561667 	 Cost: 173.846971
Iter: 840 	 Weight: -0.254611 	 Bias: 23.555118 	 Cost: 173.761125
Iter: 845 	 Weight: -0.254450 	 Bias: 23.548570 	 Cost: 173.675322
Iter: 850 	 Weight: -0.254288 	 Bias: 23.542023 	 Cost: 173.589561
Iter: 855 	 Weight: -0.254127 	 Bias: 23.535478 	 Cost: 173.503842
Iter: 860 	 Weight: -0.253965 	 Bias: 23.528935 	 Cost: 173.418166
Iter: 865 	 Weight: -0.253804 	 Bias: 23.522394 	 Cost: 173.332533
Iter: 870 	 Weight: -0.253642 	 Bias: 23.515854 	 Cost: 173.246942
Iter: 875 	 Weight: -0.253481 	 Bias: 23.509316 	 Cost: 173.161394
Iter: 880 	 Weight: -0.253319 	 Bias: 23.502779 	 Cost: 173.075888
Iter: 885 	 Weight: -0.253158 	 Bias: 23.496244 	 Cost: 172.990425
Iter: 890 	 Weight: -0.252997 	 Bias: 23.489710 	 Cost: 172.905004
Iter: 895 	 Weight: -0.252835 	 Bias: 23.483179 	 Cost: 172.819625
Iter: 900 	 Weight: -0.252674 	 Bias: 23.476648 	 Cost: 172.734289
Iter: 905 	 Weight: -0.252513 	 Bias: 23.470120 	 Cost: 172.648995
Iter: 910 	 Weight: -0.252352 	 Bias: 23.463593 	 Cost: 172.563744
Iter: 915 	 Weight: -0.252191 	 Bias: 23.457068 	 Cost: 172.478535
Iter: 920 	 Weight: -0.252030 	 Bias: 23.450544 	 Cost: 172.393368
Iter: 925 	 Weight: -0.251869 	 Bias: 23.444022 	 Cost: 172.308244
Iter: 930 	 Weight: -0.251708 	 Bias: 23.437501 	 Cost: 172.223161
Iter: 935 	 Weight: -0.251547 	 Bias: 23.430983 	 Cost: 172.138121
Iter: 940 	 Weight: -0.251386 	 Bias: 23.424465 	 Cost: 172.053124
Iter: 945 	 Weight: -0.251225 	 Bias: 23.417950 	 Cost: 171.968168
Iter: 950 	 Weight: -0.251064 	 Bias: 23.411436 	 Cost: 171.883255
Iter: 955 	 Weight: -0.250904 	 Bias: 23.404924 	 Cost: 171.798383
Iter: 960 	 Weight: -0.250743 	 Bias: 23.398413 	 Cost: 171.713554
Iter: 965 	 Weight: -0.250582 	 Bias: 23.391904 	 Cost: 171.628767
Iter: 970 	 Weight: -0.250421 	 Bias: 23.385396 	 Cost: 171.544022
Iter: 975 	 Weight: -0.250261 	 Bias: 23.378890 	 Cost: 171.459320
Iter: 980 	 Weight: -0.250100 	 Bias: 23.372386 	 Cost: 171.374659
Iter: 985 	 Weight: -0.249940 	 Bias: 23.365883 	 Cost: 171.290040
Iter: 990 	 Weight: -0.249779 	 Bias: 23.359382 	 Cost: 171.205464
Iter: 995 	 Weight: -0.249619 	 Bias: 23.352883 	 Cost: 171.120929
Iter: 1000 	 Weight: -0.249458 	 Bias: 23.346385 	 Cost: 171.036436
Iter: 1005 	 Weight: -0.249298 	 Bias: 23.339889 	 Cost: 170.951985
Iter: 1010 	 Weight: -0.249138 	 Bias: 23.333394 	 Cost: 170.867577
Iter: 1015 	 Weight: -0.248977 	 Bias: 23.326901 	 Cost: 170.783210
Iter: 1020 	 Weight: -0.248817 	 Bias: 23.320410 	 Cost: 170.698885
Iter: 1025 	 Weight: -0.248657 	 Bias: 23.313920 	 Cost: 170.614602
Iter: 1030 	 Weight: -0.248497 	 Bias: 23.307432 	 Cost: 170.530360
Iter: 1035 	 Weight: -0.248337 	 Bias: 23.300945 	 Cost: 170.446161
Iter: 1040 	 Weight: -0.248177 	 Bias: 23.294460 	 Cost: 170.362003
Iter: 1045 	 Weight: -0.248016 	 Bias: 23.287977 	 Cost: 170.277887
Iter: 1050 	 Weight: -0.247856 	 Bias: 23.281495 	 Cost: 170.193813
Iter: 1055 	 Weight: -0.247696 	 Bias: 23.275015 	 Cost: 170.109781
Iter: 1060 	 Weight: -0.247537 	 Bias: 23.268537 	 Cost: 170.025790
Iter: 1065 	 Weight: -0.247377 	 Bias: 23.262060 	 Cost: 169.941841
Iter: 1070 	 Weight: -0.247217 	 Bias: 23.255585 	 Cost: 169.857934
Iter: 1075 	 Weight: -0.247057 	 Bias: 23.249111 	 Cost: 169.774068
Iter: 1080 	 Weight: -0.246897 	 Bias: 23.242639 	 Cost: 169.690244
Iter: 1085 	 Weight: -0.246737 	 Bias: 23.236169 	 Cost: 169.606462
Iter: 1090 	 Weight: -0.246578 	 Bias: 23.229700 	 Cost: 169.522721
Iter: 1095 	 Weight: -0.246418 	 Bias: 23.223232 	 Cost: 169.439022
Iter: 1100 	 Weight: -0.246259 	 Bias: 23.216767 	 Cost: 169.355364
Iter: 1105 	 Weight: -0.246099 	 Bias: 23.210303 	 Cost: 169.271748
Iter: 1110 	 Weight: -0.245939 	 Bias: 23.203840 	 Cost: 169.188174
Iter: 1115 	 Weight: -0.245780 	 Bias: 23.197380 	 Cost: 169.104640
Iter: 1120 	 Weight: -0.245620 	 Bias: 23.190920 	 Cost: 169.021149
Iter: 1125 	 Weight: -0.245461 	 Bias: 23.184463 	 Cost: 168.937699
Iter: 1130 	 Weight: -0.245302 	 Bias: 23.178007 	 Cost: 168.854290
Iter: 1135 	 Weight: -0.245142 	 Bias: 23.171552 	 Cost: 168.770923
Iter: 1140 	 Weight: -0.244983 	 Bias: 23.165100 	 Cost: 168.687597
Iter: 1145 	 Weight: -0.244824 	 Bias: 23.158648 	 Cost: 168.604312
Iter: 1150 	 Weight: -0.244665 	 Bias: 23.152199 	 Cost: 168.521069
Iter: 1155 	 Weight: -0.244505 	 Bias: 23.145751 	 Cost: 168.437867
Iter: 1160 	 Weight: -0.244346 	 Bias: 23.139304 	 Cost: 168.354707
Iter: 1165 	 Weight: -0.244187 	 Bias: 23.132860 	 Cost: 168.271588
Iter: 1170 	 Weight: -0.244028 	 Bias: 23.126416 	 Cost: 168.188510
Iter: 1175 	 Weight: -0.243869 	 Bias: 23.119975 	 Cost: 168.105473
Iter: 1180 	 Weight: -0.243710 	 Bias: 23.113535 	 Cost: 168.022477
Iter: 1185 	 Weight: -0.243551 	 Bias: 23.107096 	 Cost: 167.939523
Iter: 1190 	 Weight: -0.243392 	 Bias: 23.100660 	 Cost: 167.856610
Iter: 1195 	 Weight: -0.243233 	 Bias: 23.094224 	 Cost: 167.773738
Iter: 1200 	 Weight: -0.243074 	 Bias: 23.087791 	 Cost: 167.690907
Iter: 1205 	 Weight: -0.242916 	 Bias: 23.081359 	 Cost: 167.608118
Iter: 1210 	 Weight: -0.242757 	 Bias: 23.074928 	 Cost: 167.525369
Iter: 1215 	 Weight: -0.242598 	 Bias: 23.068500 	 Cost: 167.442662
Iter: 1220 	 Weight: -0.242440 	 Bias: 23.062072 	 Cost: 167.359995
Iter: 1225 	 Weight: -0.242281 	 Bias: 23.055647 	 Cost: 167.277370
Iter: 1230 	 Weight: -0.242122 	 Bias: 23.049223 	 Cost: 167.194785
Iter: 1235 	 Weight: -0.241964 	 Bias: 23.042800 	 Cost: 167.112242
Iter: 1240 	 Weight: -0.241805 	 Bias: 23.036380 	 Cost: 167.029740
Iter: 1245 	 Weight: -0.241647 	 Bias: 23.029960 	 Cost: 166.947278
Iter: 1250 	 Weight: -0.241488 	 Bias: 23.023543 	 Cost: 166.864858
Iter: 1255 	 Weight: -0.241330 	 Bias: 23.017127 	 Cost: 166.782478
Iter: 1260 	 Weight: -0.241172 	 Bias: 23.010712 	 Cost: 166.700140
Iter: 1265 	 Weight: -0.241013 	 Bias: 23.004299 	 Cost: 166.617842
Iter: 1270 	 Weight: -0.240855 	 Bias: 22.997888 	 Cost: 166.535585
Iter: 1275 	 Weight: -0.240697 	 Bias: 22.991478 	 Cost: 166.453369
Iter: 1280 	 Weight: -0.240539 	 Bias: 22.985070 	 Cost: 166.371194
Iter: 1285 	 Weight: -0.240380 	 Bias: 22.978664 	 Cost: 166.289060
Iter: 1290 	 Weight: -0.240222 	 Bias: 22.972259 	 Cost: 166.206966
Iter: 1295 	 Weight: -0.240064 	 Bias: 22.965856 	 Cost: 166.124913
Iter: 1300 	 Weight: -0.239906 	 Bias: 22.959454 	 Cost: 166.042901
Iter: 1305 	 Weight: -0.239748 	 Bias: 22.953054 	 Cost: 165.960930
Iter: 1310 	 Weight: -0.239590 	 Bias: 22.946655 	 Cost: 165.878999
Iter: 1315 	 Weight: -0.239432 	 Bias: 22.940258 	 Cost: 165.797109
Iter: 1320 	 Weight: -0.239274 	 Bias: 22.933863 	 Cost: 165.715260
Iter: 1325 	 Weight: -0.239117 	 Bias: 22.927469 	 Cost: 165.633451
Iter: 1330 	 Weight: -0.238959 	 Bias: 22.921077 	 Cost: 165.551683
Iter: 1335 	 Weight: -0.238801 	 Bias: 22.914686 	 Cost: 165.469955
Iter: 1340 	 Weight: -0.238643 	 Bias: 22.908297 	 Cost: 165.388268
Iter: 1345 	 Weight: -0.238486 	 Bias: 22.901910 	 Cost: 165.306622
Iter: 1350 	 Weight: -0.238328 	 Bias: 22.895524 	 Cost: 165.225016
Iter: 1355 	 Weight: -0.238170 	 Bias: 22.889140 	 Cost: 165.143451
Iter: 1360 	 Weight: -0.238013 	 Bias: 22.882757 	 Cost: 165.061926
Iter: 1365 	 Weight: -0.237855 	 Bias: 22.876376 	 Cost: 164.980442
Iter: 1370 	 Weight: -0.237698 	 Bias: 22.869996 	 Cost: 164.898998
Iter: 1375 	 Weight: -0.237540 	 Bias: 22.863618 	 Cost: 164.817594
Iter: 1380 	 Weight: -0.237383 	 Bias: 22.857242 	 Cost: 164.736231
Iter: 1385 	 Weight: -0.237226 	 Bias: 22.850867 	 Cost: 164.654909
Iter: 1390 	 Weight: -0.237068 	 Bias: 22.844494 	 Cost: 164.573627
Iter: 1395 	 Weight: -0.236911 	 Bias: 22.838123 	 Cost: 164.492385
Iter: 1400 	 Weight: -0.236754 	 Bias: 22.831753 	 Cost: 164.411183
Iter: 1405 	 Weight: -0.236596 	 Bias: 22.825384 	 Cost: 164.330022
Iter: 1410 	 Weight: -0.236439 	 Bias: 22.819017 	 Cost: 164.248901
Iter: 1415 	 Weight: -0.236282 	 Bias: 22.812652 	 Cost: 164.167820
Iter: 1420 	 Weight: -0.236125 	 Bias: 22.806288 	 Cost: 164.086780
Iter: 1425 	 Weight: -0.235968 	 Bias: 22.799926 	 Cost: 164.005780
Iter: 1430 	 Weight: -0.235811 	 Bias: 22.793566 	 Cost: 163.924820
Iter: 1435 	 Weight: -0.235654 	 Bias: 22.787207 	 Cost: 163.843900
Iter: 1440 	 Weight: -0.235497 	 Bias: 22.780849 	 Cost: 163.763020
Iter: 1445 	 Weight: -0.235340 	 Bias: 22.774494 	 Cost: 163.682181
Iter: 1450 	 Weight: -0.235183 	 Bias: 22.768139 	 Cost: 163.601382
Iter: 1455 	 Weight: -0.235026 	 Bias: 22.761787 	 Cost: 163.520623
Iter: 1460 	 Weight: -0.234870 	 Bias: 22.755436 	 Cost: 163.439904
Iter: 1465 	 Weight: -0.234713 	 Bias: 22.749086 	 Cost: 163.359225
Iter: 1470 	 Weight: -0.234556 	 Bias: 22.742738 	 Cost: 163.278586
Iter: 1475 	 Weight: -0.234400 	 Bias: 22.736392 	 Cost: 163.197987
Iter: 1480 	 Weight: -0.234243 	 Bias: 22.730047 	 Cost: 163.117428
Iter: 1485 	 Weight: -0.234086 	 Bias: 22.723704 	 Cost: 163.036909
Iter: 1490 	 Weight: -0.233930 	 Bias: 22.717362 	 Cost: 162.956430
Iter: 1495 	 Weight: -0.233773 	 Bias: 22.711022 	 Cost: 162.875991
Iter: 1500 	 Weight: -0.233617 	 Bias: 22.704684 	 Cost: 162.795592
Iter: 1505 	 Weight: -0.233460 	 Bias: 22.698347 	 Cost: 162.715233
Iter: 1510 	 Weight: -0.233304 	 Bias: 22.692012 	 Cost: 162.634914
Iter: 1515 	 Weight: -0.233148 	 Bias: 22.685678 	 Cost: 162.554635
Iter: 1520 	 Weight: -0.232991 	 Bias: 22.679346 	 Cost: 162.474395
Iter: 1525 	 Weight: -0.232835 	 Bias: 22.673015 	 Cost: 162.394196
Iter: 1530 	 Weight: -0.232679 	 Bias: 22.666686 	 Cost: 162.314036
Iter: 1535 	 Weight: -0.232523 	 Bias: 22.660359 	 Cost: 162.233916
Iter: 1540 	 Weight: -0.232366 	 Bias: 22.654033 	 Cost: 162.153836
Iter: 1545 	 Weight: -0.232210 	 Bias: 22.647709 	 Cost: 162.073795
Iter: 1550 	 Weight: -0.232054 	 Bias: 22.641386 	 Cost: 161.993795
Iter: 1555 	 Weight: -0.231898 	 Bias: 22.635065 	 Cost: 161.913834
Iter: 1560 	 Weight: -0.231742 	 Bias: 22.628745 	 Cost: 161.833913
Iter: 1565 	 Weight: -0.231586 	 Bias: 22.622427 	 Cost: 161.754031
Iter: 1570 	 Weight: -0.231430 	 Bias: 22.616111 	 Cost: 161.674189
Iter: 1575 	 Weight: -0.231274 	 Bias: 22.609796 	 Cost: 161.594387
Iter: 1580 	 Weight: -0.231118 	 Bias: 22.603482 	 Cost: 161.514624
Iter: 1585 	 Weight: -0.230963 	 Bias: 22.597171 	 Cost: 161.434901
Iter: 1590 	 Weight: -0.230807 	 Bias: 22.590860 	 Cost: 161.355218
Iter: 1595 	 Weight: -0.230651 	 Bias: 22.584552 	 Cost: 161.275574
Iter: 1600 	 Weight: -0.230495 	 Bias: 22.578245 	 Cost: 161.195969
Iter: 1605 	 Weight: -0.230340 	 Bias: 22.571939 	 Cost: 161.116405
Iter: 1610 	 Weight: -0.230184 	 Bias: 22.565635 	 Cost: 161.036879
Iter: 1615 	 Weight: -0.230028 	 Bias: 22.559333 	 Cost: 160.957393
Iter: 1620 	 Weight: -0.229873 	 Bias: 22.553032 	 Cost: 160.877947
Iter: 1625 	 Weight: -0.229717 	 Bias: 22.546733 	 Cost: 160.798540
Iter: 1630 	 Weight: -0.229562 	 Bias: 22.540435 	 Cost: 160.719173
Iter: 1635 	 Weight: -0.229407 	 Bias: 22.534139 	 Cost: 160.639845
Iter: 1640 	 Weight: -0.229251 	 Bias: 22.527845 	 Cost: 160.560556
Iter: 1645 	 Weight: -0.229096 	 Bias: 22.521552 	 Cost: 160.481307
Iter: 1650 	 Weight: -0.228940 	 Bias: 22.515260 	 Cost: 160.402097
Iter: 1655 	 Weight: -0.228785 	 Bias: 22.508970 	 Cost: 160.322926
Iter: 1660 	 Weight: -0.228630 	 Bias: 22.502682 	 Cost: 160.243795
Iter: 1665 	 Weight: -0.228475 	 Bias: 22.496395 	 Cost: 160.164702
Iter: 1670 	 Weight: -0.228320 	 Bias: 22.490110 	 Cost: 160.085650
Iter: 1675 	 Weight: -0.228164 	 Bias: 22.483827 	 Cost: 160.006636
Iter: 1680 	 Weight: -0.228009 	 Bias: 22.477545 	 Cost: 159.927662
Iter: 1685 	 Weight: -0.227854 	 Bias: 22.471264 	 Cost: 159.848727
Iter: 1690 	 Weight: -0.227699 	 Bias: 22.464985 	 Cost: 159.769831
Iter: 1695 	 Weight: -0.227544 	 Bias: 22.458708 	 Cost: 159.690974
Iter: 1700 	 Weight: -0.227389 	 Bias: 22.452432 	 Cost: 159.612157
Iter: 1705 	 Weight: -0.227235 	 Bias: 22.446158 	 Cost: 159.533378
Iter: 1710 	 Weight: -0.227080 	 Bias: 22.439885 	 Cost: 159.454639
Iter: 1715 	 Weight: -0.226925 	 Bias: 22.433614 	 Cost: 159.375939
Iter: 1720 	 Weight: -0.226770 	 Bias: 22.427344 	 Cost: 159.297277
Iter: 1725 	 Weight: -0.226615 	 Bias: 22.421076 	 Cost: 159.218655
Iter: 1730 	 Weight: -0.226461 	 Bias: 22.414810 	 Cost: 159.140072
Iter: 1735 	 Weight: -0.226306 	 Bias: 22.408545 	 Cost: 159.061528
Iter: 1740 	 Weight: -0.226151 	 Bias: 22.402282 	 Cost: 158.983023
Iter: 1745 	 Weight: -0.225997 	 Bias: 22.396020 	 Cost: 158.904557
Iter: 1750 	 Weight: -0.225842 	 Bias: 22.389760 	 Cost: 158.826130
Iter: 1755 	 Weight: -0.225688 	 Bias: 22.383501 	 Cost: 158.747742
Iter: 1760 	 Weight: -0.225533 	 Bias: 22.377244 	 Cost: 158.669393
Iter: 1765 	 Weight: -0.225379 	 Bias: 22.370988 	 Cost: 158.591082
Iter: 1770 	 Weight: -0.225224 	 Bias: 22.364734 	 Cost: 158.512811
Iter: 1775 	 Weight: -0.225070 	 Bias: 22.358482 	 Cost: 158.434578
Iter: 1780 	 Weight: -0.224916 	 Bias: 22.352231 	 Cost: 158.356384
Iter: 1785 	 Weight: -0.224761 	 Bias: 22.345981 	 Cost: 158.278229
Iter: 1790 	 Weight: -0.224607 	 Bias: 22.339734 	 Cost: 158.200113
Iter: 1795 	 Weight: -0.224453 	 Bias: 22.333487 	 Cost: 158.122036
Iter: 1800 	 Weight: -0.224299 	 Bias: 22.327243 	 Cost: 158.043997
Iter: 1805 	 Weight: -0.224145 	 Bias: 22.320999 	 Cost: 157.965998
Iter: 1810 	 Weight: -0.223991 	 Bias: 22.314758 	 Cost: 157.888036
Iter: 1815 	 Weight: -0.223837 	 Bias: 22.308518 	 Cost: 157.810114
Iter: 1820 	 Weight: -0.223683 	 Bias: 22.302279 	 Cost: 157.732230
Iter: 1825 	 Weight: -0.223529 	 Bias: 22.296042 	 Cost: 157.654385
Iter: 1830 	 Weight: -0.223375 	 Bias: 22.289807 	 Cost: 157.576579
Iter: 1835 	 Weight: -0.223221 	 Bias: 22.283573 	 Cost: 157.498811
Iter: 1840 	 Weight: -0.223067 	 Bias: 22.277341 	 Cost: 157.421082
Iter: 1845 	 Weight: -0.222913 	 Bias: 22.271110 	 Cost: 157.343392
Iter: 1850 	 Weight: -0.222759 	 Bias: 22.264881 	 Cost: 157.265740
Iter: 1855 	 Weight: -0.222606 	 Bias: 22.258653 	 Cost: 157.188126
Iter: 1860 	 Weight: -0.222452 	 Bias: 22.252427 	 Cost: 157.110551
Iter: 1865 	 Weight: -0.222298 	 Bias: 22.246202 	 Cost: 157.033015
Iter: 1870 	 Weight: -0.222145 	 Bias: 22.239979 	 Cost: 156.955517
Iter: 1875 	 Weight: -0.221991 	 Bias: 22.233758 	 Cost: 156.878058
Iter: 1880 	 Weight: -0.221837 	 Bias: 22.227538 	 Cost: 156.800637
Iter: 1885 	 Weight: -0.221684 	 Bias: 22.221319 	 Cost: 156.723254
Iter: 1890 	 Weight: -0.221530 	 Bias: 22.215102 	 Cost: 156.645910
Iter: 1895 	 Weight: -0.221377 	 Bias: 22.208887 	 Cost: 156.568605
Iter: 1900 	 Weight: -0.221224 	 Bias: 22.202673 	 Cost: 156.491337
Iter: 1905 	 Weight: -0.221070 	 Bias: 22.196461 	 Cost: 156.414108
Iter: 1910 	 Weight: -0.220917 	 Bias: 22.190250 	 Cost: 156.336918
Iter: 1915 	 Weight: -0.220764 	 Bias: 22.184041 	 Cost: 156.259766
Iter: 1920 	 Weight: -0.220610 	 Bias: 22.177834 	 Cost: 156.182652
Iter: 1925 	 Weight: -0.220457 	 Bias: 22.171628 	 Cost: 156.105576
Iter: 1930 	 Weight: -0.220304 	 Bias: 22.165423 	 Cost: 156.028539
Iter: 1935 	 Weight: -0.220151 	 Bias: 22.159220 	 Cost: 155.951540
Iter: 1940 	 Weight: -0.219998 	 Bias: 22.153019 	 Cost: 155.874579
Iter: 1945 	 Weight: -0.219845 	 Bias: 22.146819 	 Cost: 155.797656
Iter: 1950 	 Weight: -0.219692 	 Bias: 22.140620 	 Cost: 155.720771
Iter: 1955 	 Weight: -0.219539 	 Bias: 22.134424 	 Cost: 155.643925
Iter: 1960 	 Weight: -0.219386 	 Bias: 22.128228 	 Cost: 155.567117
Iter: 1965 	 Weight: -0.219233 	 Bias: 22.122034 	 Cost: 155.490347
Iter: 1970 	 Weight: -0.219080 	 Bias: 22.115842 	 Cost: 155.413615
Iter: 1975 	 Weight: -0.218927 	 Bias: 22.109652 	 Cost: 155.336921
Iter: 1980 	 Weight: -0.218774 	 Bias: 22.103462 	 Cost: 155.260266
Iter: 1985 	 Weight: -0.218622 	 Bias: 22.097275 	 Cost: 155.183648
Iter: 1990 	 Weight: -0.218469 	 Bias: 22.091089 	 Cost: 155.107068
Iter: 1995 	 Weight: -0.218316 	 Bias: 22.084904 	 Cost: 155.030527

iters = np.arange(2000)
# because the derivation of cost is too large
# for visulazation, we need to shrink cost
cost_history = np.log(np.array(cost_history))
plt.plot(iters, cost_history)

[]

sales_predict = weight_final*radio_sample + bias_final
plt.scatter(radio_sample, sales_sample, alpha=0.5)
plt.plot(radio_sample, sales_predict, c="r")
plt.xlabel("radio")
plt.ylabel("sales")

Text(0, 0.5, 'sales')

The setting of hyperparameters is very import for gradient descent method. There must be lots of tricks which is another topic.

Task 2

Single feature —> multiple features

scalar —> matrix(vector)

Generate data

sample_size = 200
feature_size = 3
np.random.seed(5)
feature_sample = 50*np.random.rand(feature_size)*np.random.rand(sample_size, feature_size)

np.random.seed(123)
weight_sample = np.random.rand(feature_size).reshape(feature_size, 1)

np.random.seed(888)
noise = np.random.normal(0, 0.01, sample_size).reshape(sample_size ,1)

result_sample = np.dot(feature, weight) + noise

Normalization

We want to shrink the data to reduce the time to change weight. There must be lots of tricks which is another topic.

def normalize(feature):
    #feature: sample size * feature size
    feature = feature.astype("float64")
    sample_size, feature_size = feature.shape
    for i in range(feature_size):
        fmean = np.mean(feature[:, i])
        frange = np.amax(feature[:, i]) - np.amin(feature[:, i])
        feature[:, i] = (feature[:, i] - fmean) / frange
    return feature

Prediction format

For simplifing the problem, here we just think bias is zero.

$Y_{sample*1}=F_{sample*feature} \cdot W_{feature*1}$

def predict(features, weight):
    weight = weight.reshape(len(weight), 1)
    prediction = np.dot(features, weight)
    return prediction

Cost function

$MSE=\frac{1}{2 N} \left\|Y_{target}-Y_{predict}\right\|_2^2$

def cost_function(features, targets, weight):
    #here weights should be (feature, 1)
    weight = weight.reshape(len(weight), 1)
    targets = targets.reshape(len(targets), 1)
    error = ((targets - predict(features, weight))**2).sum()
    return error/(2.0*len(targets))

Gradient descent

Just the equation derived in the watermelon book(3.11)

def update_weight(features, targets, weight, learning_rate):
    weight = weight.reshape(len(weight), 1)
    targets = targets.reshape(len(targets), 1)
    gradient = np.dot(features.T, (predict(features, weight) - targets))
    gradient = gradient / len(targets)
    weight = weight - gradient*learning_rate
    return weight

Training

def train(features, targets, weight, learning_rate, iters):
    cost_history = []
    for i in range(iters):
        weight = update_weight(features, targets, weight, learning_rate)
        cost = cost_function(features, targets, weight)
        cost_history.append(cost)
        if i % 5 == 0:
            print("Iter: %d \t \t Cost: %4f" %(i, cost))
    return weight, cost_history

Model evaluation

weight_initial = np.array([0, 0, 0])
weight, cost_history = train(feature_sample, result_sample, weight_initial, 0.0001, 1000)

Iter: 0 	 	 Cost: 64.953149
Iter: 5 	 	 Cost: 33.915984
Iter: 10 	 	 Cost: 18.615390
Iter: 15 	 	 Cost: 11.053208
Iter: 20 	 	 Cost: 7.296763
Iter: 25 	 	 Cost: 5.412361
Iter: 30 	 	 Cost: 4.449195
Iter: 35 	 	 Cost: 3.939745
Iter: 40 	 	 Cost: 3.654115
Iter: 45 	 	 Cost: 3.479253
Iter: 50 	 	 Cost: 3.359556
Iter: 55 	 	 Cost: 3.267672
Iter: 60 	 	 Cost: 3.190135
Iter: 65 	 	 Cost: 3.120312
Iter: 70 	 	 Cost: 3.054928
Iter: 75 	 	 Cost: 2.992358
Iter: 80 	 	 Cost: 2.931792
Iter: 85 	 	 Cost: 2.872817
Iter: 90 	 	 Cost: 2.815220
Iter: 95 	 	 Cost: 2.758881
Iter: 100 	 	 Cost: 2.703732
Iter: 105 	 	 Cost: 2.649725
Iter: 110 	 	 Cost: 2.596827
Iter: 115 	 	 Cost: 2.545010
Iter: 120 	 	 Cost: 2.494250
Iter: 125 	 	 Cost: 2.444522
Iter: 130 	 	 Cost: 2.395806
Iter: 135 	 	 Cost: 2.348080
Iter: 140 	 	 Cost: 2.301324
Iter: 145 	 	 Cost: 2.255518
Iter: 150 	 	 Cost: 2.210643
Iter: 155 	 	 Cost: 2.166678
Iter: 160 	 	 Cost: 2.123606
Iter: 165 	 	 Cost: 2.081409
Iter: 170 	 	 Cost: 2.040067
Iter: 175 	 	 Cost: 1.999565
Iter: 180 	 	 Cost: 1.959884
Iter: 185 	 	 Cost: 1.921007
Iter: 190 	 	 Cost: 1.882919
Iter: 195 	 	 Cost: 1.845603
Iter: 200 	 	 Cost: 1.809044
Iter: 205 	 	 Cost: 1.773225
Iter: 210 	 	 Cost: 1.738131
Iter: 215 	 	 Cost: 1.703749
Iter: 220 	 	 Cost: 1.670062
Iter: 225 	 	 Cost: 1.637057
Iter: 230 	 	 Cost: 1.604721
Iter: 235 	 	 Cost: 1.573038
Iter: 240 	 	 Cost: 1.541997
Iter: 245 	 	 Cost: 1.511583
Iter: 250 	 	 Cost: 1.481784
Iter: 255 	 	 Cost: 1.452588
Iter: 260 	 	 Cost: 1.423981
Iter: 265 	 	 Cost: 1.395952
Iter: 270 	 	 Cost: 1.368490
Iter: 275 	 	 Cost: 1.341582
Iter: 280 	 	 Cost: 1.315217
Iter: 285 	 	 Cost: 1.289384
Iter: 290 	 	 Cost: 1.264073
Iter: 295 	 	 Cost: 1.239272
Iter: 300 	 	 Cost: 1.214971
Iter: 305 	 	 Cost: 1.191160
Iter: 310 	 	 Cost: 1.167828
Iter: 315 	 	 Cost: 1.144967
Iter: 320 	 	 Cost: 1.122566
Iter: 325 	 	 Cost: 1.100617
Iter: 330 	 	 Cost: 1.079109
Iter: 335 	 	 Cost: 1.058034
Iter: 340 	 	 Cost: 1.037383
Iter: 345 	 	 Cost: 1.017147
Iter: 350 	 	 Cost: 0.997318
Iter: 355 	 	 Cost: 0.977888
Iter: 360 	 	 Cost: 0.958848
Iter: 365 	 	 Cost: 0.940190
Iter: 370 	 	 Cost: 0.921908
Iter: 375 	 	 Cost: 0.903992
Iter: 380 	 	 Cost: 0.886435
Iter: 385 	 	 Cost: 0.869231
Iter: 390 	 	 Cost: 0.852372
Iter: 395 	 	 Cost: 0.835851
Iter: 400 	 	 Cost: 0.819660
Iter: 405 	 	 Cost: 0.803795
Iter: 410 	 	 Cost: 0.788246
Iter: 415 	 	 Cost: 0.773010
Iter: 420 	 	 Cost: 0.758078
Iter: 425 	 	 Cost: 0.743444
Iter: 430 	 	 Cost: 0.729104
Iter: 435 	 	 Cost: 0.715050
Iter: 440 	 	 Cost: 0.701277
Iter: 445 	 	 Cost: 0.687779
Iter: 450 	 	 Cost: 0.674550
Iter: 455 	 	 Cost: 0.661586
Iter: 460 	 	 Cost: 0.648880
Iter: 465 	 	 Cost: 0.636427
Iter: 470 	 	 Cost: 0.624223
Iter: 475 	 	 Cost: 0.612262
Iter: 480 	 	 Cost: 0.600540
Iter: 485 	 	 Cost: 0.589050
Iter: 490 	 	 Cost: 0.577790
Iter: 495 	 	 Cost: 0.566753
Iter: 500 	 	 Cost: 0.555936
Iter: 505 	 	 Cost: 0.545334
Iter: 510 	 	 Cost: 0.534943
Iter: 515 	 	 Cost: 0.524758
Iter: 520 	 	 Cost: 0.514775
Iter: 525 	 	 Cost: 0.504991
Iter: 530 	 	 Cost: 0.495400
Iter: 535 	 	 Cost: 0.486000
Iter: 540 	 	 Cost: 0.476786
Iter: 545 	 	 Cost: 0.467754
Iter: 550 	 	 Cost: 0.458901
Iter: 555 	 	 Cost: 0.450224
Iter: 560 	 	 Cost: 0.441718
Iter: 565 	 	 Cost: 0.433380
Iter: 570 	 	 Cost: 0.425207
Iter: 575 	 	 Cost: 0.417196
Iter: 580 	 	 Cost: 0.409342
Iter: 585 	 	 Cost: 0.401644
Iter: 590 	 	 Cost: 0.394097
Iter: 595 	 	 Cost: 0.386700
Iter: 600 	 	 Cost: 0.379448
Iter: 605 	 	 Cost: 0.372339
Iter: 610 	 	 Cost: 0.365369
Iter: 615 	 	 Cost: 0.358537
Iter: 620 	 	 Cost: 0.351839
Iter: 625 	 	 Cost: 0.345273
Iter: 630 	 	 Cost: 0.338836
Iter: 635 	 	 Cost: 0.332525
Iter: 640 	 	 Cost: 0.326338
Iter: 645 	 	 Cost: 0.320272
Iter: 650 	 	 Cost: 0.314326
Iter: 655 	 	 Cost: 0.308495
Iter: 660 	 	 Cost: 0.302779
Iter: 665 	 	 Cost: 0.297175
Iter: 670 	 	 Cost: 0.291680
Iter: 675 	 	 Cost: 0.286292
Iter: 680 	 	 Cost: 0.281010
Iter: 685 	 	 Cost: 0.275831
Iter: 690 	 	 Cost: 0.270753
Iter: 695 	 	 Cost: 0.265774
Iter: 700 	 	 Cost: 0.260892
Iter: 705 	 	 Cost: 0.256105
Iter: 710 	 	 Cost: 0.251411
Iter: 715 	 	 Cost: 0.246808
Iter: 720 	 	 Cost: 0.242295
Iter: 725 	 	 Cost: 0.237870
Iter: 730 	 	 Cost: 0.233530
Iter: 735 	 	 Cost: 0.229275
Iter: 740 	 	 Cost: 0.225102
Iter: 745 	 	 Cost: 0.221010
Iter: 750 	 	 Cost: 0.216997
Iter: 755 	 	 Cost: 0.213062
Iter: 760 	 	 Cost: 0.209202
Iter: 765 	 	 Cost: 0.205418
Iter: 770 	 	 Cost: 0.201706
Iter: 775 	 	 Cost: 0.198066
Iter: 780 	 	 Cost: 0.194496
Iter: 785 	 	 Cost: 0.190995
Iter: 790 	 	 Cost: 0.187561
Iter: 795 	 	 Cost: 0.184194
Iter: 800 	 	 Cost: 0.180891
Iter: 805 	 	 Cost: 0.177651
Iter: 810 	 	 Cost: 0.174474
Iter: 815 	 	 Cost: 0.171358
Iter: 820 	 	 Cost: 0.168301
Iter: 825 	 	 Cost: 0.165303
Iter: 830 	 	 Cost: 0.162362
Iter: 835 	 	 Cost: 0.159477
Iter: 840 	 	 Cost: 0.156648
Iter: 845 	 	 Cost: 0.153873
Iter: 850 	 	 Cost: 0.151150
Iter: 855 	 	 Cost: 0.148479
Iter: 860 	 	 Cost: 0.145860
Iter: 865 	 	 Cost: 0.143290
Iter: 870 	 	 Cost: 0.140768
Iter: 875 	 	 Cost: 0.138295
Iter: 880 	 	 Cost: 0.135869
Iter: 885 	 	 Cost: 0.133489
Iter: 890 	 	 Cost: 0.131153
Iter: 895 	 	 Cost: 0.128862
Iter: 900 	 	 Cost: 0.126615
Iter: 905 	 	 Cost: 0.124410
Iter: 910 	 	 Cost: 0.122246
Iter: 915 	 	 Cost: 0.120123
Iter: 920 	 	 Cost: 0.118040
Iter: 925 	 	 Cost: 0.115997
Iter: 930 	 	 Cost: 0.113992
Iter: 935 	 	 Cost: 0.112024
Iter: 940 	 	 Cost: 0.110094
Iter: 945 	 	 Cost: 0.108199
Iter: 950 	 	 Cost: 0.106340
Iter: 955 	 	 Cost: 0.104516
Iter: 960 	 	 Cost: 0.102726
Iter: 965 	 	 Cost: 0.100970
Iter: 970 	 	 Cost: 0.099246
Iter: 975 	 	 Cost: 0.097555
Iter: 980 	 	 Cost: 0.095895
Iter: 985 	 	 Cost: 0.094265
Iter: 990 	 	 Cost: 0.092667
Iter: 995 	 	 Cost: 0.091097

iters = np.arange(1000)
plt.plot(iters, cost_history)

[]

由回归到分类

In this task, I just use the watermelon dataset which is shown below.

Logistic Regression ( Binary Classification）的实际编码

Dataset

import numpy as np
import seaborn as sns
import pandas as pd
import matplotlib.pyplot as plt
import math

def createDataSet():
    """
    创建测试的数据集，里面的数值中具有连续值
    :return:
    """
    dataSet = [
        # 1
        ['青绿', '蜷缩', '浊响', '清晰', '凹陷', '硬滑', 0.697, 0.460, '好瓜'],
        # 2
        ['乌黑', '蜷缩', '沉闷', '清晰', '凹陷', '硬滑', 0.774, 0.376, '好瓜'],
        # 3
        ['乌黑', '蜷缩', '浊响', '清晰', '凹陷', '硬滑', 0.634, 0.264, '好瓜'],
        # 4
        ['青绿', '蜷缩', '沉闷', '清晰', '凹陷', '硬滑', 0.608, 0.318, '好瓜'],
        # 5
        ['浅白', '蜷缩', '浊响', '清晰', '凹陷', '硬滑', 0.556, 0.215, '好瓜'],
        # 6
        ['青绿', '稍蜷', '浊响', '清晰', '稍凹', '软粘', 0.403, 0.237, '好瓜'],
        # 7
        ['乌黑', '稍蜷', '浊响', '稍糊', '稍凹', '软粘', 0.481, 0.149, '好瓜'],
        # 8
        ['乌黑', '稍蜷', '浊响', '清晰', '稍凹', '硬滑', 0.437, 0.211, '好瓜'],

        # ----------------------------------------------------
        # 9
        ['乌黑', '稍蜷', '沉闷', '稍糊', '稍凹', '硬滑', 0.666, 0.091, '坏瓜'],
        # 10
        ['青绿', '硬挺', '清脆', '清晰', '平坦', '软粘', 0.243, 0.267, '坏瓜'],
        # 11
        ['浅白', '硬挺', '清脆', '模糊', '平坦', '硬滑', 0.245, 0.057, '坏瓜'],
        # 12
        ['浅白', '蜷缩', '浊响', '模糊', '平坦', '软粘', 0.343, 0.099, '坏瓜'],
        # 13
        ['青绿', '稍蜷', '浊响', '稍糊', '凹陷', '硬滑', 0.639, 0.161, '坏瓜'],
        # 14
        ['浅白', '稍蜷', '沉闷', '稍糊', '凹陷', '硬滑', 0.657, 0.198, '坏瓜'],
        # 15
        ['乌黑', '稍蜷', '浊响', '清晰', '稍凹', '软粘', 0.360, 0.370, '坏瓜'],
        # 16
        ['浅白', '蜷缩', '浊响', '模糊', '平坦', '硬滑', 0.593, 0.042, '坏瓜'],
        # 17
        ['青绿', '蜷缩', '沉闷', '稍糊', '稍凹', '硬滑', 0.719, 0.103, '坏瓜']
    ]
    return dataSet

dataSet = createDataSet()
dataSet = np.array(dataSet)[:, 6:]
dataSet[dataSet == '好瓜'] = 1
dataSet[dataSet == '坏瓜'] = 0
dataSet = dataSet.astype('float64')

dataSet

array([[0.697, 0.46 , 1.   ],
       [0.774, 0.376, 1.   ],
       [0.634, 0.264, 1.   ],
       [0.608, 0.318, 1.   ],
       [0.556, 0.215, 1.   ],
       [0.403, 0.237, 1.   ],
       [0.481, 0.149, 1.   ],
       [0.437, 0.211, 1.   ],
       [0.666, 0.091, 0.   ],
       [0.243, 0.267, 0.   ],
       [0.245, 0.057, 0.   ],
       [0.343, 0.099, 0.   ],
       [0.639, 0.161, 0.   ],
       [0.657, 0.198, 0.   ],
       [0.36 , 0.37 , 0.   ],
       [0.593, 0.042, 0.   ],
       [0.719, 0.103, 0.   ]])

Visulization

data_in_frame = pd.DataFrame(data=dataSet, columns=["density", "sugar_ratio","label"])

data_in_frame

	density	sugar_ratio	label
0	0.697	0.460	1.0
1	0.774	0.376	1.0
2	0.634	0.264	1.0
3	0.608	0.318	1.0
4	0.556	0.215	1.0
5	0.403	0.237	1.0
6	0.481	0.149	1.0
7	0.437	0.211	1.0
8	0.666	0.091	0.0
9	0.243	0.267	0.0
10	0.245	0.057	0.0
11	0.343	0.099	0.0
12	0.639	0.161	0.0
13	0.657	0.198	0.0
14	0.360	0.370	0.0
15	0.593	0.042	0.0
16	0.719	0.103	0.0

sns.scatterplot(data=data_in_frame, x='density', y="sugar_ratio", hue="label")

Prediction Format

\begin{split}p \geq 0.5, class=1 \
p < 0.5, class=0\end{split}

\begin{split}P(class=1) = \frac{1} {1 + e^{-z}}\end{split}

def sigmoid(x, derivative=False):
    sigm = 1. / (1. + np.exp(-x))
    if derivative:
        return sigm * (1. - sigm)
    return sigm

def predict(features, weights):
    """
    features: sample size * feature size
    weights: feature size * 1
    """
    weights = weights.reshape(len(weights),1)
    z = np.dot(features, weights)
    return sigmoid(z)

How could we do here? From Watermelon_book, we know that we can use MLE to estimate the parameters.

Meanwhile we can also repeat what we did in last experiment where we just included a cost function and optimized the patameters by decrease the cost function.

Cost function

We can still use MSE as a cost function. But here what we use is called cross-entroy function actually. The reason why we abandon the previous one is another topic. Basicly, it’s because of non-linear tranformation.

\begin{split}-{(y\log§ + (1 - y)\log(1 - p))}\end{split}

def cost_function(features, weights, labels):
    weights = weights.reshape(len(weights),1)
    labels = labels.reshape(len(labels),1)
    y = predict(features, weights)
    class1_cost = -labels*np.log(y)
    class2_cost = -labels*np.log(y)
    cost = (class1_cost + class2_cost).sum()/len(labels)
    return cost

Gradient descent

\begin{align}
s’(z) & = s(z)(1 - s(z))
\end{align}

\begin{split}C’ = x(s(z) - y)\end{split}

def update_weights(features, weights, labels, learning_rate):
    weights = weights.reshape(len(weights),1)
    labels = labels.reshape(len(labels),1)
    p = predict(features, weights)
    gradient = np.dot(features.T, p - labels) / len(labels)
    weights = weights - gradient*learning_rate
    return weights

Training

def training(features, weights, labels, learning_rate, iters):
    cost_history = []
    for i in range(iters):
        weights = update_weights(features, weights, labels, learning_rate)
        cost = cost_function(features, weights, labels)
        cost_history.append(cost)
        if i%1000 == 0:
            print("iters is %d \t \t cost is %f"%(i, cost))
    return weights, cost_history

weights_initial = np.array([0, 0])
weights, cost_history = training(dataSet[:, :2], weights_initial, dataSet[:, -1], 0.1, 20000)

iters is 0 	 	 cost is 0.651950
iters is 1000 	 	 cost is 0.572514
iters is 2000 	 	 cost is 0.546478
iters is 3000 	 	 cost is 0.528388
iters is 4000 	 	 cost is 0.515437
iters is 5000 	 	 cost is 0.505936
iters is 6000 	 	 cost is 0.498829
iters is 7000 	 	 cost is 0.493431
iters is 8000 	 	 cost is 0.489282
iters is 9000 	 	 cost is 0.486062
iters is 10000 	 	 cost is 0.483545
iters is 11000 	 	 cost is 0.481564
iters is 12000 	 	 cost is 0.479999
iters is 13000 	 	 cost is 0.478757
iters is 14000 	 	 cost is 0.477769
iters is 15000 	 	 cost is 0.476981
iters is 16000 	 	 cost is 0.476351
iters is 17000 	 	 cost is 0.475847
iters is 18000 	 	 cost is 0.475443
iters is 19000 	 	 cost is 0.475119

Model evaluation

Visulization

iters = np.arange(20000)
plt.plot(iters, cost_history)

[]

From the fig abobe, we can think the cost function may still decrease when we update the weights furthur.

Mapping probabilities to classes

def classify(predictions):
    predictions[predictions >= 0.5 ] = 1
    predictions[predictions < 0.5 ] = 0
    return predictions
def accuracy(predictions, labels):
    predictions = predictions.astype('int').reshape(len(predictions,))
    labels = labels.astype('int')
    diff = np.abs(predictions - labels)
    same = len(labels) - diff.sum()
    return same/len(labels)

predicted_label = classify(predict(dataSet[:, :2], weights))

accuracy(predicted_label, dataSet[:, -1])

0.8235294117647058

你可能感兴趣的:([Watermelon_book] Chapter 3 Linear Model)

斤斤计较的婚姻到底有多难？白心之岂必有为
很多人私聊我会问到在哪个人群当中斤斤计较的人最多？我都会回答他，一般婚姻出现问题的斤斤计较的人士会非常多，以我多年经验，在婚姻落的一塌糊涂的人当中，斤斤计较的人数占比在20～30%以上，也就是说10个婚姻出现问题的斤斤计较的人有2-3个有多不减。在婚姻出问题当中，有大量的心理不平衡的、尖酸刻薄的怨妇。在婚姻中仅斤斤计较有两种类型：第一种是物质上的，另一种是精神上的。在物质与精神上抠门已经严重的影响
情绪觉察日记第37天露露_e800
今天是家庭关系规划师的第二阶最后一天，慧萍老师帮我做了个案，帮我处理了埋在心底好多年的一份恐惧，并给了我深深的力量！这几天出来学习，爸妈过来婆家帮我带小孩，妈妈出于爱帮我收拾东西，并跟我先生和婆婆产生矛盾，妈妈觉得他们没有照顾好我…。今晚回家见到妈妈，我很欣赏她并赞扬她，妈妈说今晚要跟我睡我说好，当我们俩躺在床上准备睡觉的时候，我握着妈妈的手对她说:妈妈这几天辛苦你了，你看你多利害把我们的家收拾得
QQ群采集助手，精准引流必备神器 2401_87347160 其他经验分享
功能概述微信群查找与筛选工具是一款专为微信用户设计的辅助工具，它通过关键词搜索功能，帮助用户快速找到相关的微信群，并提供筛选是否需要验证的群组的功能。主要功能关键词搜索：用户可以输入关键词，工具将自动查找包含该关键词的微信群。筛选功能：工具提供筛选机制，用户可以选择是否只显示需要验证或不需要验证的群组。精准引流：通过上述功能，用户可以更精准地找到目标群组，进行有效的引流操作。3.设备需求该工具可以
底层逆袭到底有多难，不甘平凡的你准备好了吗？让吴起给你说说造命者说
底层逆袭到底有多难，不甘平凡的你准备好了吗？让吴起给你说说我叫吴起，生于公元前440年的战国初期，正是群雄并起、天下纷争不断的时候。后人说我是军事家、政治家、改革家，是兵家代表人物。评价我一生历仕鲁、魏、楚三国，通晓兵家、法家、儒家三家思想，在内政军事上都有极高的成就。周安王二十一年（公元前381年），因变法得罪守旧贵族，被人乱箭射死。我出生在卫国一个“家累万金”的富有家庭，从年轻时候起就不甘平凡
【iOS】MVC设计模式 Magnetic_h ios mvc 设计模式 objective-c 学习 ui
MVC前言如何设计一个程序的结构，这是一门专门的学问，叫做"架构模式"（architecturalpattern），属于编程的方法论。MVC模式就是架构模式的一种。它是Apple官方推荐的App开发架构，也是一般开发者最先遇到、最经典的架构。MVC各层controller层Controller/ViewController/VC（控制器）负责协调Model和View，处理大部分逻辑它将数据从Mod
谢谢你们，爱你们！鹿游儿
昨天家人去泡温泉，二个孩子也带着去，出发前一晚，匆匆下班，赶回家和孩子一起收拾。饭后，我拿出笔和本子（上次去澳门时做手帐的本子）写下了1\2\3\4\5\6\7\8\9,让后让小壹去思考，带什么出发去旅游呢？她在对应的数字旁边画上了，泳衣、泳圈、肖恩、内衣内裤、tapuy、拖鞋……画完后，就让她自己对着这个本子，将要带的，一一带上，没想到这次带的书还是这本《便便工厂》(晚上姑婆发照片过来，妹妹累得
2021年12月19日，春蕾教育集团团建活动感受——黄晓丹黄错错加油
感受:1.从陌生到熟悉的过程。游戏环节让我们在轻松的氛围中得到了锻炼，也增长了不少知识。2.游戏过程中，我们贡献的是个人力量，展现的是团队的力量。它磨合的往往不止是工作的熟悉，更是观念上契合度的贴近。3.这和工作是一样的道理。在各自的岗位上，每个人摆正自己的位置、各司其职充分发挥才能，并团结一致劲往一处使，才能实现最大的成功。新知:1.团队精神需要不断地创新。过去，人们把创新看作是冒风险，现在人们
《策划经理回忆录之二》路基雅虎
话说三年变六年，飘了，飘了……眨眼，2013年5月，老吴回到了他的家乡——油城从新开启他的工作幻想症生涯。很庆幸，这是一家很有追求，同时敢于尝试的，且实力不容低调的新星房企——金源置业(前身泰源置业)更值得庆幸的是第一个盘就是油城十路的标杆之一:金源盛世。2013年5月，到2015年11月，两年的陪伴，迎来了一场大爆发。2000个筹，5万/筹，直接回笼1个亿！！！这……让我开始认真审视这座看似五线
Long类型前后端数据不一致 igotyback 前端
响应给前端的数据浏览器控制台中response中看到的Long类型的数据是正常的到前端数据不一致前后端数据类型不匹配是一个常见问题，尤其是当后端使用Java的Long类型（64位）与前端JavaScript的Number类型（最大安全整数为2^53-1，即16位）进行数据交互时，很容易出现精度丢失的问题。这是因为JavaScript中的Number类型无法安全地表示超过16位的整数。为了解决这个问
swagger访问路径 igotyback swagger
Swagger2.x版本访问地址：http://{ip}:{port}/{context-path}/swagger-ui.html{ip}是你的服务器IP地址。{port}是你的应用服务端口，通常为8080。{context-path}是你的应用上下文路径，如果应用部署在根路径下，则为空。Swagger3.x版本对于Swagger3.x版本（也称为OpenAPI3）访问地址：http://{ip
30天风格练习-DAY2 黄希夷
Day2（重义）在一个周日/一周的最后一天，我来到位于市中心/市区繁华地带的一家购物中心/商场，中心内人很多/熙熙攘攘。我注意到/看见一个独行/孤身一人的年轻女孩/，留着一头引人注目/长过腰际的头发，上身穿一件暗红色/比正红色更深的衣服/穿在身体上的东西。走下扶梯的时候，她摔倒了/跌向地面，在她正要站起来/让身体离开地面的时候，过长/超过一般人长度的头发被支撑身体/躯干的手掌压/按在下面，她赶紧用
ArcGIS栅格计算器常见公式（赋值、0和空值的转换、补充栅格空值）研学随笔 arcgis 经验分享
我们在使用ArcGIS时通常经常用到栅格计算器，今天主要给大家介绍我日常中经常用到的几个公式，供大家参考学习。将特定值（-9999）赋值为0，例如-9999.Con("raster"==-9999,0,"raster")2.给空值赋予特定的值（如0）Con(IsNull("raster"),0,"raster")3.将特定的栅格值(如1)赋值为空值，其他保留原值SetNull("raster"==
高级编程--XML+socket练习题 masa010 java 开发语言
1.北京华北2114.8万人上海华东2,500万人广州华南1292.68万人成都华西1417万人（1）使用dom4j将信息存入xml中（2）读取信息，并打印控制台（3）添加一个city节点与子节点（4）使用socketTCP协议编写服务端与客户端，客户端输入城市ID，服务器响应相应城市信息（5）使用socketTCP协议编写服务端与客户端，客户端要求用户输入city对象，服务端接收并使用dom4j
开心蒋泳频
从无比抗拒来上课到接受，感动，收获～看着波哥成长，晶晶幸福笑容满面。感觉自己做的事情很有意义，很开心！还有3个感召目标就是还有三个有缘人，哈哈。明天感召去明日计划：8：30-11：00小公益11：00-21点上班，感召图片发自App图片发自App图片发自App
2018-07-23-催眠日作业-#不一样的31天#-66小鹿小鹿_33
预言日：人总是在逃避命运的路上，与之不期而遇。心理学上有个著名的名词，叫做自证预言；经济学上也有一个很著名的定律叫做，墨菲定律；在灵修派上，还有一个很著名的法则，叫做吸引力法则。这3个领域的词，虽然看起来不太一样，但是他们都在告诉人们一个现象：你越担心什么，就越有可能会发生什么。同样的道理，你越想得到什么，就应该要积极地去创造什么。无论是自证预言，墨菲定律还是吸引力法则，对人都有正反2个维度的影响
本周第二次约练 2cfbdfe28a51
中原焦点团队中24初26刘霞2021.12.3约练161次，分享第368天当事人虽然是带着问题来的，但是咨询过程中发现，她是经过自己不断地调整和努力才走到现在的，看到当事人的不容易，找到例外，发现资源，力量感也就随之而来。增强画面感，或者说重温，会给当事人带来更深刻的感受。
回溯 Leetcode 332 重新安排行程 mmaerd Leetcode刷题学习记录 leetcode 算法职场和发展
重新安排行程Leetcode332学习记录自代码随想录给你一份航线列表tickets，其中tickets[i]=[fromi,toi]表示飞机出发和降落的机场地点。请你对该行程进行重新规划排序。所有这些机票都属于一个从JFK（肯尼迪国际机场）出发的先生，所以该行程必须从JFK开始。如果存在多种有效的行程，请你按字典排序返回最小的行程组合。例如，行程[“JFK”,“LGA”]与[“JFK”,“LGB
今日联对0306 诗图佳得
自对联：烟销皓月临江浒，水漫金山荡塔裙。一一肖士平2020.3.6.1、试对肖老师联：烟销皓月临江浒，夜笼寒沙梦晚舟。耀哥求正2、试对萧老师联:烟销浩月临江浒，雾散乾坤解汉城。秀霞习作请各位老师校正3、自对联：烟销皓月临江浒，水漫金山荡塔裙。一一肖士平2020.3.6.4、试对肖老师垫场联：烟销皓月临江浒，雾锁寒林缈葉丛。小智求正[抱拳]5、试对肖老师联：烟销皓月临江浒；风卷乱云入峰巅。一一五品6
每日一题——第八十九题互联网打工人no1 C语言程序设计每日一练 c语言
题目：在字符串中找到提取数字，并统计一共找到多少整数，a123xxyu23&8889，那么找到的整数为123，23，8889//思想：#include#include#includeintmain(){charstr[]="a123xxyu23&8889";intcount=0;intnum=0;//用于临时存放当前正在构建的整数。boolinNum=false;//用于标记当前是否正在读取一个整
每日一题——第八十四题互联网打工人no1 C语言程序设计每日一练 c语言
题目：编写函数1、输入10个职工的姓名和职工号2、按照职工由大到小顺序排列，姓名顺序也随之调整3、要求输入一个职工号，用折半查找法找出该职工的姓名#define_CRT_SECURE_NO_WARNINGS#include#include#defineMAX_EMPLOYEES10typedefstruct{intid;charname[50];}Empolyee;voidinputEmploye
《小满细雨轻湿尘》快乐的人ZZM
图片发自App《小满细雨轻湿尘》文/快乐的人zzm小满细雨轻湿尘石榴花开落纷纷落红不是无情物坠入泥土育养根2018-5-23
WPF中的ComboBox控件几种数据绑定的方式互联网打工人no1 wpf c#
一、用字典给ItemsSource赋值（此绑定用的地方很多，建议熟练掌握）在XMAL中：在CS文件中privatevoidBindData(){DictionarydicItem=newDictionary();dicItem.add(1,"北京");dicItem.add(2,"上海");dicItem.add(3,"广州");cmb_list.ItemsSource=dicItem;cmb_l
Python中os.environ基本介绍及使用方法鹤冲天Pro #Python python 服务器开发语言
文章目录python中os.environos.environ简介os.environ进行环境变量的增删改查python中os.environ的使用详解1.简介2.key字段详解2.1常见key字段3.os.environ.get()用法4.环境变量的增删改查和判断是否存在4.1新增环境变量4.2更新环境变量4.3获取环境变量4.4删除环境变量4.5判断环境变量是否存在python中os.envi
怎么起诉借钱不还的人？怎样起诉欠款不还的人？影子爱学习
怎么起诉借钱不还的人？怎样起诉欠款不还的人？如果遇到难以解决的法律问题，我们可以匹配专业律师。例如：婚姻家庭（离婚纠纷）、刑事辩护、合同纠纷、债权债务、房产（继承）纠纷、交通事故、劳动争议、人身损害、公司相关法律事务（法律顾问）等咨询推荐手机/微信:15633770876【全国案件皆可】借钱不还起诉对方需要哪些资料起诉欠钱不还的，一般需要的材料包括以下这些：借据、收据、欠条、付款凭证等证据，以及向
2019-12-22-22:30 涓涓1016
今天是冬至，写下我的日更，是因为这两天的学习真的是能量的满满，让我看到了自己，未来另外一种可能性，也让我看到了这两年这几年的过程中我所接受那些痛苦的来源。一切的根源和痛苦都来自于人生，家庭，而你的原生家庭，你的爸爸和妈妈，是因为你这个灵魂在那一刻选择他们作为你的爸爸和妈妈来的，所以你得接受他，你得接纳他，他就是因为他的存在而给你的学习和成长带来这些痛苦，那其实是你必然要经历的这个过程，当你去接纳的
linux sdl windows.h,Windows下的SDL安装奔跑吧linux内核 linux sdl windows.h
首先你要下载并安装SDL开发包。如果装在C盘下，路径为C:\SDL1.2.5如果在WINDOWS下。你可以按以下步骤：1.打开VC++，点击"Tools",Options2,点击directories选项3.选择"Includefiles"增加一个新的路径。"C:\SDL1.2.5\include"4，现在选择"Libaryfiles“增加"C:\SDL1.2.5\lib"现在你可以开始编写你的第
Python教程：一文了解使用Python处理XPath 旦莫 Python进阶 python 开发语言
目录1.环境准备1.1安装lxml1.2验证安装2.XPath基础2.1什么是XPath？2.2XPath语法2.3示例XML文档3.使用lxml解析XML3.1解析XML文档3.2查看解析结果4.XPath查询4.1基本路径查询4.2使用属性查询4.3查询多个节点5.XPath的高级用法5.1使用逻辑运算符5.2使用函数6.实战案例6.1从网页抓取数据6.1.1安装Requests库6.1.2代
相信相信的力量孙丽_cdb3
孙丽中级十期坚持分享第345天有一个特别有哲理的故事：有一只老鹰下了蛋，这个蛋，不知怎的就滚到了鸡窝里去了，鸡也下了一窝蛋，然后鸡妈妈把这些蛋全都浮出来了，孵出来之后等小鸡长大一点了，就觉得鹰蛋孵出来的那只小鹰怪模怪样，这些小鸡都嘲笑它，真难看，真笨，丑死了，那只小鹰觉得自己真是谁也不像，真是不好看，后来鸡妈妈也不喜欢他，我怎么生出你这样的孩子来了？真烦人，后来这群小鸡和小鹰一起生活，有一天，老鹰
感赏日志133 马姐读书
图片发自App感赏自己今天买个扫地机，以后可以解放出来多看点书，让这个智能小机器人替我工作了。感赏孩子最近进步很大，每天按时上学，认真听课，认真背书，主动认真完成老师布置的作业。感赏自己明白自己容易受到某人的影响，心情不好，每当此刻我就会舒缓，感赏，让自己尽快抽离，想好的一面。感赏儿子今天在我提醒他事情时，告诉我谢谢妈妈对我的提醒我明白了，而不是说我啰嗦，管事情，孩子更懂事了，懂得感恩了。投射父母
基于社交网络算法优化的二维最大熵图像分割智能算法研学社（Jack旭）智能优化算法应用图像分割算法 php 开发语言
智能优化算法应用：基于社交网络优化的二维最大熵图像阈值分割-附代码文章目录智能优化算法应用：基于社交网络优化的二维最大熵图像阈值分割-附代码1.前言2.二维最大熵阈值分割原理3.基于社交网络优化的多阈值分割4.算法结果：5.参考文献：6.Matlab代码摘要：本文介绍基于最大熵的图像分割，并且应用社交网络算法进行阈值寻优。1.前言阅读此文章前，请阅读《图像分割：直方图区域划分及信息统计介绍》htt
jsonp 常用util方法 hw1287789687 jsonp jsonp常用方法 jsonp callback
jsonp 常用java方法 (1)以jsonp的形式返回:函数名(json字符串) /*** * 用于jsonp调用 * @param map : 用于构造json数据 * @param callback : 回调的javascript方法名 * @param filters : <code>SimpleBeanPropertyFilter theFilt
多线程场景 alafqq 多线程
0 能不能简单描述一下你在java web开发中需要用到多线程编程的场景？0 对多线程有些了解，但是不太清楚具体的应用场景，能简单说一下你遇到的多线程编程的场景吗？ Java多线程 2012年11月23日 15:41 Young9007 Young9007 4 0 0 4 Comment添加评论关注(2) 3个答案按时间排序按投票排序 0 0 最典型的如： 1、
Maven学习——修改Maven的本地仓库路径 Kai_Ge maven
安装Maven后我们会在用户目录下发现.m2 文件夹。默认情况下，该文件夹下放置了Maven本地仓库.m2/repository。所有的Maven构件(artifact)都被存储到该仓库中，以方便重用。但是windows用户的操作系统都安装在C盘，把Maven仓库放到C盘是很危险的，为此我们需要修改Maven的本地仓库路径。
placeholder的浏览器兼容 120153216 placeholder
【前言】自从html5引入placeholder后，问题就来了，不支持html5的浏览器也先有这样的效果，各种兼容，之前考虑，今天测试人员逮住不放，想了个解决办法，看样子还行，记录一下。【原理】不使用placeholder，而是模拟placeholder的效果，大概就是用focus和focusout效果。【代码】 <scrip
debian_用iso文件创建本地apt源 2002wmj Debian
1.将N个debian-506-amd64-DVD-N.iso存放于本地或其他媒介内，本例是放在本机/iso/目录下 2.创建N个挂载点目录如下： debian:~#mkdir –r /media/dvd1 debian:~#mkdir –r /media/dvd2 debian:~#mkdir –r /media/dvd3 …. debian:~#mkdir –r /media
SQLSERVER耗时最长的SQL 357029540 SQL Server
对于DBA来说，经常要知道存储过程的某些信息： 1. 执行了多少次 2. 执行的执行计划如何 3. 执行的平均读写如何 4. 执行平均需要多少时间列名 &
com/genuitec/eclipse/j2eedt/core/J2EEProjectUtil 7454103 eclipse
今天eclipse突然报了com/genuitec/eclipse/j2eedt/core/J2EEProjectUtil 错误，并且工程文件打不开了，在网上找了一下资料，然后按照方法操作了一遍，好了，解决方法如下：错误提示信息： An error has occurred.See error log for more details. Reason: com/genuitec/
用正则删除文本中的html标签 adminjun java html 正则表达式去掉html标签
使用文本编辑器录入文章存入数据中的文本是HTML标签格式，由于业务需要对HTML标签进行去除只保留纯净的文本内容，于是乎Java实现自动过滤。如下： public static String Html2Text(String inputString) { String htmlStr = inputString; // 含html标签的字符串 String textSt
嵌入式系统设计中常用总线和接口 aijuans linux 基础
嵌入式系统设计中常用总线和接口任何一个微处理器都要与一定数量的部件和外围设备连接，但如果将各部件和每一种外围设备都分别用一组线路与CPU直接连接，那么连线
Java函数调用方式——按值传递 ayaoxinchao java 按值传递对象基础数据类型
Java使用按值传递的函数调用方式，这往往使我感到迷惑。因为在基础数据类型和对象的传递上，我就会纠结于到底是按值传递，还是按引用传递。其实经过学习，Java在任何地方，都一直发挥着按值传递的本色。首先，让我们看一看基础数据类型是如何按值传递的。 public static void main(String[] args) { int a = 2;
ios音量线性下降 bewithme ios音量
直接上代码吧 //second 几秒内下降为0 - (void)reduceVolume:(int)second { KGVoicePlayer *player = [KGVoicePlayer defaultPlayer]; if (!_flag) { _tempVolume = player.volume;
与其怨它不如爱它 bijian1013 选择理想职业规划
抱怨工作是年轻人的常态，但爱工作才是积极的心态，与其怨它不如爱它。一般来说，在公司干了一两年后，不少年轻人容易产生怨言，除了具体的埋怨公司“扭门”，埋怨上司无能以外，也有许多人是因为根本不爱自已的那份工作，工作完全成了谋生的手段，跟自已的性格、专业、爱好都相差甚远。
一边时间不够用一边浪费时间 bingyingao 工作时间浪费
一方面感觉时间严重不够用，另一方面又在不停的浪费时间。每一个周末，晚上熬夜看电影到凌晨一点，早上起不来一直睡到10点钟，10点钟起床，吃饭后玩手机到下午一点。精神还是很差，下午像一直野鬼在城市里晃荡。为何不尝试晚上10点钟就睡，早上7点就起，时间完全是一样的，把看电影的时间换到早上，精神好，气色好，一天好状态。控制让自己周末早睡早起，你就成功了一半。有多少个工作
【Scala八】Scala核心二：隐式转换 bit1129 scala
Implicits work like this: if you call a method on a Scala object, and the Scala compiler does not see a definition for that method in the class definition for that object, the compiler will try to con
sudoku slover in Haskell (2) bookjovi haskell sudoku
继续精简haskell版的sudoku程序，稍微改了一下，这次用了8行，同时性能也提高了很多，对每个空格的所有解不是通过尝试算出来的，而是直接得出。 board = [0,3,4,1,7,0,5,0,0, 0,6,0,0,0,8,3,0,1, 7,0,0,3,0,0,0,0,6, 5,0,0,6,4,0,8,0,7,
Java-Collections Framework学习与总结-HashSet和LinkedHashSet BrokenDreams linkedhashset
本篇总结一下两个常用的集合类HashSet和LinkedHashSet。它们都实现了相同接口java.util.Set。Set表示一种元素无序且不可重复的集合；之前总结过的java.util.List表示一种元素可重复且有序
读《研磨设计模式》-代码笔记-备忘录模式-Memento bylijinnan java 设计模式
声明：本文只为方便我个人查阅和理解，详细的分析以及源代码请移步原作者的博客http://chjavach.iteye.com/ import java.util.ArrayList; import java.util.List; /* * 备忘录模式的功能是，在不破坏封装性的前提下，捕获一个对象的内部状态，并在对象之外保存这个状态，为以后的状态恢复作“备忘”
《RAW格式照片处理专业技法》笔记 cherishLC PS
注意，这不是教程！仅记录楼主之前不太了解的一、色彩（空间）管理作者建议采用ProRGB（色域最广），但camera raw中设为ProRGB，而PS中则在ProRGB的基础上，将gamma值设为了1.8（更符合人眼）注意：bridge、camera raw怎么设置显示、输出的颜色都是正确的（会读取文件内的颜色配置文件），但用PS输出jpg文件时，必须先用Edit->conv
使用 Git 下载 Spring 源码编译 for Eclipse crabdave eclipse
使用 Git 下载 Spring 源码编译 for Eclipse 1、安装gradle，下载 http://www.gradle.org/downloads 配置环境变量GRADLE_HOME，配置PATH %GRADLE_HOME%/bin，cmd，gradle -v 2、spring4 用jdk8 下载 https://jdk8.java.
mysql连接拒绝问题 daizj mysql 登录权限
mysql中在其它机器连接mysql服务器时报错问题汇总一、[running][email protected]:~$mysql -uroot -h 192.168.9.108 -p //带-p参数，在下一步进行密码输入 Enter password: //无字符串输入 ERROR 1045 (28000): Access
Google Chrome 为何打压 H.264 dsjt apple html5 chrome Google
Google 今天在 Chromium 官方博客宣布由于 H.264 编解码器并非开放标准，Chrome 将在几个月后正式停止对 H.264 视频解码的支持，全面采用开放的 WebM 和 Theora 格式。 Google 在博客上表示，自从 WebM 视频编解码器推出以后，在性能、厂商支持以及独立性方面已经取得了很大的进步，为了与 Chromium 现有支持的編解码器保持一致，Chrome
yii 获取控制器名和方法名 dcj3sjt126com yii framework
1. 获取控制器名在控制器中获取控制器名: $name = $this->getId(); 在视图中获取控制器名: $name = Yii::app()->controller->id; 2. 获取动作名在控制器beforeAction()回调函数中获取动作名: $name =
Android知识总结（二） come_for_dream android
明天要考试了，速速总结如下 1、Activity的启动模式 standard：每次调用Activity的时候都创建一个（可以有多个相同的实例，也允许多个相同Activity叠加。） singleTop：可以有多个实例，但是不允许多个相同Activity叠加。即，如果Ac
高洛峰收徒第二期：寻找未来的“技术大牛” ——折腾一年，奖励20万元 gcq511120594 工作项目管理
高洛峰，兄弟连IT教育合伙人、猿代码创始人、PHP培训第一人、《细说PHP》作者、软件开发工程师、《IT峰播》主创人、PHP讲师的鼻祖！首期现在的进程刚刚过半，徒弟们真的很棒，人品都没的说，团结互助，学习刻苦，工作认真积极，灵活上进。我几乎会把他们全部留下来，现在已有一多半安排了实际的工作，并取得了很好的成绩。等他们出徒之日，凭他们的能力一定能够拿到高薪，而且我还承诺过一个徒弟，当他拿到大学毕
linux expect heipark expect
1. 创建、编辑文件go.sh #!/usr/bin/expect spawn sudo su admin expect "*password*" { send "13456\r\n" } interact 2. 设置权限 chmod u+x go.sh 3.
Spring4.1新特性——静态资源处理增强 jinnianshilongnian spring 4.1
目录 Spring4.1新特性——综述 Spring4.1新特性——Spring核心部分及其他 Spring4.1新特性——Spring缓存框架增强 Spring4.1新特性——异步调用和事件机制的异常处理 Spring4.1新特性——数据库集成测试脚本初始化 Spring4.1新特性——Spring MVC增强 Spring4.1新特性——页面自动化测试框架Spring MVC T
idea ubuntuxia 乱码 liyonghui160com
1.首先需要在windows字体目录下或者其它地方找到simsun.ttf 这个字体文件。 2.在ubuntu 下可以执行下面操作安装该字体： sudo mkdir /usr/share/fonts/truetype/simsun sudo cp simsun.ttf /usr/share/fonts/truetype/simsun fc-cache -f -v
改良程序的11技巧 pda158 技巧
有很多理由都能说明为什么我们应该写出清晰、可读性好的程序。最重要的一点，程序你只写一次，但以后会无数次的阅读。当你第二天回头来看你的代码时，你就要开始阅读它了。当你把代码拿给其他人看时，他必须阅读你的代码。因此，在编写时多花一点时间，你会在阅读它时节省大量的时间。让我们看一些基本的编程技巧：尽量保持方法简短永远永远不要把同一个变量用于多个不同的
300个涵盖IT各方面的免费资源（下）——工作与学习篇 shoothao 创业免费资源学习课程远程工作
工作与生产效率: A. 背景声音 Noisli:背景噪音与颜色生成器。 Noizio:环境声均衡器。 Defonic:世界上任何的声响都可混合成美丽的旋律。 Designers.mx:设计者为设计者所准备的播放列表。 Coffitivity:这里的声音就像咖啡馆里放的一样。 B. 避免注意力分散 Self Co
深入浅出RPC uule rpc
深入浅出RPC-浅出篇深入浅出RPC-深入篇 RPC Remote Procedure Call Protocol 远程过程调用协议它是一种通过网络从远程计算机程序上请求服务，而不需要了解底层网络技术的协议。RPC协议假定某些传输协议的存在，如TCP或UDP，为通信程序之间携带信息数据。在OSI网络通信模型中，RPC跨越了传输层和应用层。RPC使得开发