汉澳sinox2016安装tensorflow,scikit-learn,theano做机器学习

汉澳sinox2016能不能安装运行机器学习软件呢？作为领先的操作系统，不能进行人工智能开发，无论如何也说不过去。现在就来运行安装流行的机器学习软件。

经典的机器学习软件主要是tensorflow,scikit-learn,theano。

怎么安装呢？其实不是很难的，因为有了软件库，用的是freebsd的软件库。先设置/etc/pkg/FreeBSD.conf,打开后编辑如下后保存。

#Sinox_install_cdrom: {
#  url: "file:///usr/packages",
#  mirror_type: "none",
#  enabled: yes
#}

#Sinox: {
#  enabled: no
#}


FreeBSD: {
#  url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly",
    url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}

真正目的是让安装软件包指向最新的pkg库，然后安装

安装tensorflow

命令

pkg install py27-tensorflow

只有python2.7!还没有python3.5版本的

他会安装新版本依赖软件包。卸载旧版本，这个可能会导致有些软件包被卸载，但是不重新安装，所以可能导致有的软件不能用！

安装好了以后运行python命令

输入

import tensorflow as tf

没提示，运行成功

安装 sckit-learn

pkg install py27-scikit-learn

root@www:~ # pkg install py27-scipy
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 2 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        py27-scipy: 0.19.0_2
        swig13: 1.3.40_1

用pip install sckit-learn不成功，因为可能不对应操作系统版本

安装 theano

pkg install py27-theano


New packages to be INSTALLED:
        py27-theano: 0.9.0_1

现在测试这些工具是否安装好

python

>>> import sklearn
>>> import theano
>>> import numpy
>>> import scipy
>>> import tensorflow
>>>

>>> quit()

pip命令查看安装的python包

root@www:~ # pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
attrs (17.1.0)
backports.weakref (1.0rc1)
bleach (1.4.2)
cffi (1.7.0)
characteristic (14.3.0)
cryptography (1.7.2)
enum34 (1.1.6)
funcsigs (1.0.2)
gps (3.14)
html5lib (0.9999999)
idna (2.5)
ipaddress (1.0.18)
libxml2-python (2.9.4)
Markdown (2.6.8)
mock (2.0.0)
mod-python (3.5.0)
numpy (1.13.1)
pbr (3.1.1)
pip (9.0.1)
protobuf (3.4.0)
pyasn1 (0.2.2)
pyasn1-modules (0.0.9)
pycparser (2.10)
pyOpenSSL (16.2.0)
scikit-learn (0.17)
scipy (0.19.0)
service-identity (16.0.0)
setuptools (36.2.2)
six (1.10.0)
tensorflow (1.2.1)
Theano (0.9.0)
Twisted-Core (15.2.1)
Werkzeug (0.12.2)
wheel (0.29.0)
zope.interface (4.1.3)
root@www:~ #

最后测试运行tensorflow程序test.py

#!/usr/bin/python
# -*- coding:utf-8 -*-


import tensorflow as tf
import numpy as np

# 使用 NumPy 生成假数据(phony data), 总共 100 个点.
x_data = np.float32(np.random.rand(2, 100)) # 随机输入
y_data = np.dot([0.100, 0.200], x_data) + 0.300

# 构造一个线性模型
#
b = tf.Variable(tf.zeros([1]))
W = tf.Variable(tf.random_uniform([1, 2], -1.0, 1.0))
y = tf.matmul(W, x_data) + b

# 最小化方差
loss = tf.reduce_mean(tf.square(y - y_data))
optimizer = tf.train.GradientDescentOptimizer(0.5)
train = optimizer.minimize(loss)

# 初始化变量
init = tf.initialize_all_variables()

# 启动图 (graph)
sess = tf.Session()
sess.run(init)

# 拟合平面
for step in xrange(0, 201):
    sess.run(train)
    if step % 20 == 0:
        print step, sess.run(W), sess.run(b)

# 得到最佳拟合结果 W: [[0.100  0.200]], b: [0.300]

结果如下：

root@www:~ # python test.py
WARNING:tensorflow:From /usr/local/lib/python2.7/site-packages/tensorflow/python/util/tf_should_use.py:170: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use `tf.global_variables_initializer` instead.
2017-09-02 10:43:24.267475: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 10:43:24.267498: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 10:43:24.267504: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 10:43:24.267508: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
can't determine number of CPU cores: assuming 4
can't determine number of CPU cores: assuming 4
0 [[ 0.53918093 -0.08097869]] [ 0.53843534]
20 [[ 0.15594584  0.12123938]] [ 0.31314299]
40 [[ 0.10854671  0.18340005]] [ 0.30456361]
60 [[ 0.10092636  0.19623375]] [ 0.30159572]
80 [[ 0.09993701  0.19906192]] [ 0.30055997]
100 [[ 0.09990507  0.19974208]] [ 0.30019689]
120 [[ 0.09995297  0.19992268]] [ 0.3000693]
140 [[ 0.0999809  0.1999753]] [ 0.30002442]
160 [[ 0.0999928   0.19999178]] [ 0.30000859]
180 [[ 0.09999738  0.19999719]] [ 0.30000305]
200 [[ 0.09999906  0.199999  ]] [ 0.30000108]
root@www:~ #

运行成功！

来运行更复杂的cnn.py

#!/usr/bin/python
# -*- coding:utf-8 -*-

from tensorflow.examples.tutorials.mnist.input_data import *
import tensorflow as tf

# 读取数据集，read_data_sets是一个已经封装好的方法，会去直接下载数据并且做预处理
mnist = read_data_sets("MNIST_data/", one_hot=True)

# # 定义数据，设置占位符
# 设置特征与标签的占位符，特征集是n×784维，标签集维n×10维，n是可以调节的
x = tf.placeholder(tf.float32, [None, 784])
y_ = tf.placeholder("float", [None, 10])
# 设置dropout的占位符，dropout用于防止过拟合
keep_pro = tf.placeholder("float")
# 将平铺的特征重构成28×28的图片像素维度，因为使用的是黑白图片，所以颜色通道维1,因为要取出所有数据，所以索引维-1
x_image = tf.reshape(x, [-1, 28, 28, 1])


# # 定义函数以方便构造网络
# 初始化权重,传入大小参数，truncated_normal函数使得w呈正太分布，
# stddev设置标准差为0.1。也就是说输入形状大小，输出正太分布的随机参数作为权重变量矩阵
def weight_variable(shape):
    initial = tf.truncated_normal(shape, stddev=0.1)
    return tf.Variable(initial)


# 初始化偏执项，传入矩阵大小的参数，生成该大小的值全部为0.1的矩阵
def bias_variable(shape):
    initial = tf.constant(0.1, shape=shape)
    return tf.Variable(initial)


# 定义卷基层，步长为1,周围补0，输入与输出的数据大小一样（可得到补全的圈数）
def conv2d(a, w):
    return tf.nn.conv2d(a, w, strides=[1, 1, 1, 1], padding='SAME')


# 定义池化层,kernel大小为2,步长为2,周围补0，输入与输出的数据大小一样（可得到补全的圈数）
def max_pool_2x2(a):
    return tf.nn.max_pool(a, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding="SAME")

# # 进入卷积层与池化层1
# 初始化权重，传入权重大小，窗口的大小是5×5,所以指向每个卷积层的权重也是5×5,
# 卷积层的神经元的个数是32,总共只有1个面（1个颜色通道）
w_conv1 = weight_variable([5, 5, 1, 32])
# 32个神经元就需要32个偏执项
b_conv1 = bias_variable([32])
# 将卷积层相对应的数据求内积再加上偏执项的这个线性函数，放入激励层relu中做非线性打转换，输出的大小是28×28×32
h_conv1 = tf.nn.relu(conv2d(x_image, w_conv1) + b_conv1)
# 将卷积层输出的数据传入池化层，根据函的设定，窗口大小维2×2,步长为2,输出的大小就降到来14×14×32
h_pool1 = max_pool_2x2(h_conv1)

# # 进入卷积层与池化层2
# 第2层卷积层由64个神经元，1个神经元要初始化的权重维度是5×5×32
w_conv2 = weight_variable([5, 5, 32, 64])
# 偏执项的数目和神经元的数目一样
b_conv2 = bias_variable([64])
# 将池化层1的输出与卷积层2的权重做内积再加上偏执项，然后进入激励函数relu，输出维度为14×14×64
h_conv2 = tf.nn.relu(conv2d(h_pool1, w_conv2) + b_conv2)
# 进入池化层，输出减半为7×7×64
h_pool2 = max_pool_2x2(h_conv2)
# # 进入全连接层1
# 初始化全链接层的权重，全了链接层有1024个神经元，每个都与池化层2的输出数据全部连接
w_fc1 = weight_variable([7*7*64, 1024])
# 偏执项也等于神经元的个数1024
b_fc1 = bias_variable([1024])
# 将池化层的输出数据拉平为1行7×7×64列打矩阵，-1表示把所有都拿出来
h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])
# 全连接计算，线性运算后再输入激励函数中，最后输出1024个数据
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, w_fc1) + b_fc1)
# 使用dropout防止过拟合
h_fc1_drop = tf.nn.dropout(h_fc1, keep_pro)

# # 进入全连接层2
# 初始化权重，全连接层2有10个神经元，上一层打输入是1024
w_fc2 = weight_variable([1024, 10])
# 偏执项为10
b_fc2 = bias_variable([10])
# 全连接的计算，然后再过一个softmax函数，输出为10个数据（10个概率）
y_conv = tf.nn.softmax(tf.matmul(h_fc1_drop, w_fc2) + b_fc2)

# # 损失函数最小的最优化计算
# 交叉熵作为目标函数计算
cross_entropy = -tf.reduce_sum(y_ * tf.log(y_conv))
# 目标函数最小训练模型，估计参数，使用的是ADAM优化器来做梯度下降
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)

# 计算预测正确的个数，tf.argmax是寻找一个tensor中每个维度的最大值所在的索引
# 因为类别是用0，1表示的，所以找出1所在打索引就能找到数字打类别
# tf.equals是检测预测与真实的标签是否一致，返回的是布尔值，true,false
correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1))
# 计算正确率,用tf.cast来将true,false转换成1,0,然后计算正确率
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

# # 创建会话,初始化变量
sess = tf.InteractiveSession()
sess.run(tf.initialize_all_variables())

# # 执行循环
for i in range(2000):
    # 每批取出50个训练样本
    batch = mnist.train.next_batch(50)
    # 循环次数是100的倍数的时候，打印东东
    if i % 100 == 0:
        # 计算正确率，
        train_accuracy = accuracy.eval(feed_dict={
           x: batch[0], y_: batch[1], keep_pro: 1.0})
        # 打印
        print "step %d, training accuracy %g" % (i, train_accuracy)
    # 执行训练模型
    train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_pro: 0.5})
# 打印测试集正确率
print "test accuracy %g" % accuracy.eval(feed_dict={
        x: mnist.test.images, y_: mnist.test.labels, keep_pro: 1.0
    })

运行后发现无法获取数据，因为被墙了。

root@www:~ # python cnn.py
Traceback (most recent call last):
  File "cnn.py", line 8, in
    mnist = read_data_sets("MNIST_data/", one_hot=True)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py", line 235, in read_data_sets
    SOURCE_URL + TRAIN_IMAGES)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py", line 208, in maybe_download
    temp_file_name, _ = urlretrieve_with_retry(source_url)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py", line 165, in wrapped_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py", line 190, in urlretrieve_with_retry
    return urllib.request.urlretrieve(url, filename)
  File "/usr/local/lib/python2.7/urllib.py", line 98, in urlretrieve
    return opener.retrieve(url, filename, reporthook, data)
  File "/usr/local/lib/python2.7/urllib.py", line 245, in retrieve
    fp = self.open(url, data)
  File "/usr/local/lib/python2.7/urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "/usr/local/lib/python2.7/urllib.py", line 443, in open_https
    h.endheaders(data)
  File "/usr/local/lib/python2.7/httplib.py", line 1053, in endheaders
    self._send_output(message_body)
  File "/usr/local/lib/python2.7/httplib.py", line 897, in _send_output
    self.send(msg)
  File "/usr/local/lib/python2.7/httplib.py", line 859, in send
    self.connect()
  File "/usr/local/lib/python2.7/httplib.py", line 1278, in connect
    server_hostname=server_hostname)
  File "/usr/local/lib/python2.7/ssl.py", line 353, in wrap_socket
    _context=self)
  File "/usr/local/lib/python2.7/ssl.py", line 601, in __init__
    self.do_handshake()
  File "/usr/local/lib/python2.7/ssl.py", line 830, in do_handshake
    self._sslobj.do_handshake()
IOError: [Errno socket error] EOF occurred in violation of protocol (_ssl.c:590)

网络问题的，不是程序的问题，再来

root@www:~ # python cnn.py
Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting MNIST_data/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
2017-09-02 11:10:59.024575: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 11:10:59.024600: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 11:10:59.024606: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 11:10:59.024610: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
can't determine number of CPU cores: assuming 4
can't determine number of CPU cores: assuming 4
WARNING:tensorflow:From /usr/local/lib/python2.7/site-packages/tensorflow/python/util/tf_should_use.py:170: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use `tf.global_variables_initializer` instead.
step 0, training accuracy 0.12
step 100, training accuracy 0.76
step 200, training accuracy 0.9
step 300, training accuracy 0.98
step 400, training accuracy 0.92
step 500, training accuracy 0.96
step 600, training accuracy 0.96
step 700, training accuracy 0.96
step 800, training accuracy 0.92
step 900, training accuracy 0.98
step 1000, training accuracy 0.98
step 1100, training accuracy 0.96
step 1200, training accuracy 0.92
step 1300, training accuracy 1
step 1400, training accuracy 1
step 1500, training accuracy 1
step 1600, training accuracy 1
step 1700, training accuracy 0.92
step 1800, training accuracy 1
step 1900, training accuracy 0.98
test accuracy 0.9773
root@www:~ #

由于最新的pkg会同步更新，所以sinox2018版本会发布最新的pkg,获得一致的应用包，同时更新最新的pkg,有惊喜。当然

主要是安装最新的机器学习软件包。

但是，你现在就可以安装使用了，何必再等呢。