【深度学习-tensorflow】CUDA + Anaconda + tensorflow + PyCharm
研究生期间的第一学年，机器学习和深度学习的氛围很浓，包括各种学术报告，各种讲座，还有老师课上的经验之谈。这学期的《机器学习》课程中，王老师提到了【深度学习】的重要性，并告诉我们，“如果不懂得深度学习，那么将来工作都不好找了”（应该是因为我们是计算机系的硕士学生，想让我们找到更好的工作）。当然，懂深度学习和用深度学习是两码事。有人说深度学习是一个“黑盒”，目前还没有严谨的数学理论支撑；计算机视觉的老师也跟我们说过，深度学习是给人们带来了方便（深度学习之前的特征很多都需要计算机视觉领域的理论来提前特征，现在深度学习在很多方面能够提取出很好的特征）并且在性能上也得到了很大的提升，但是千万不能脱离理论基础。是的，我们要学会如何使用深度学习，但同时也要掌握相关的理论知识。我的目标也是希望能够学好机器学习的理论知识，更进一步的了解【深度学习】，包括“caffe”和“tensorflow”。在写此博文时，我也还只是个刚要入门的学生，日后也将遇到各种学习和实践上的挑战，希望可以坚持整理和分享学习心得。
本文主要介绍如何在windows系统下基于Anaconda3安装深度学习框架 - GPU版的Tensorflow。本文的主要贡献如下：
配置【win7&win10 + CUDA8.0 + CUDNN5.1】的详细流程。通过本人的实践，整理了在windows平台下基于Anaconda配置“tensorflow”的详细流程，遇到的问题及其解决方案。提供了相关工具和插件的下载地址，如pycharm（包括破解方法）。通过实践，整理了一些基于tensorflow相关的案例：包括官网中tensorflow的基本用法，其他CSDN博客整理的MNIST等。
---------------------------------------------------------------
Part 1：CUDA8.0 + CUDNN5.1
这部分主要面向“GPU版本的tensorflow”，如果你只是要用CPU跑深度学习的程序，那么不用安装CUDA和CUDNN，直接根据tensorflow官网或者极客学院翻译的中文教程（“CPU”版本）进行安装即可。此外，这部分配置同样适用于另一个深度学习框架Caffe（GPU版本）的配置。下面来分享一下CUDA8.0和CUDNN5.1的安装流程：
检查下你的电脑是否有CUDA_Capable GP（独立显卡），打开cmd或者powershell执行如下语句：
control /name Microsoft.DeviceManager
英伟达独立显卡 GT 755M，比较老的独显了，实验室的是GTX 960
-- 此处介绍一款查看GPU的软件：GPU-Z
准备安装包：
1. NVIDIA CUDA Toolkit 8.0：/cuda-downloads
2. cuDNN5.1：/rdp/cudnn-download
安装NVIDIA CUDA Toolkit 8.0：
进入官网下载地址，下载过程如下一系列图示：
下载local版本的exe，local和net版本的介绍如下（摘取至官网）
Network Installer:A minimal installer which later downloads packages required for installation. Only the packages selected during the selection phase of the installer are downloaded. This installer is useful for users who want to minimize download time.
Full Installer:An installer which contains all the components of the CUDA Toolkit and does not require any further download.This installer is useful for systems which lack network access and for enterprise deployment.
直接根据推荐，选择精简版进行下载
选择“安装”NVIDIA Corporation
安装完成，安装路径默认为：C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA
配置环境变量：NVIDIA CUDA Toolkit 8.0
CUDA_PATH：C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0 CUDA_BIN_PATH：%CUDA_PATH%\bin CUDA_LIB_PATH：%CUDA_PATH%\lib\Win32CUDA_SDK_BIN：%CUDA_SDK_PATH%\bin\Win64 CUDA_SDK_LIB：%CUDA_SDK_PATH%\common\lib\x64 CUDA_SDK_PATH：C:\ProgramData\NVIDIA Corporation\CUDA Samples\v8.0在系统环境变量path后添加如下内容 %CUDA_LIB_PATH%;%CUDA_BIN_PATH%;%CUDA_SDK_LIB_PATH%;%CUDA_SDK_BIN_PATH%;
-- 输入【nvcc –V】检测是否安装成功，如下图所示：
安装cuDNN5.1：
下载cuDNN得到的是一个压缩包，将安装包解压到【C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0】的目录下，如下图所示：
选择cuDNN v5.1
cuDNN的压缩包
解压到该目录下，自动合并到相应的文件夹内
-- 至此，完成CUDA和cuDNN的安装 --
Part 2：基于Anaconda安装tensorflow
0. 安装tensorflow的核心步骤总结【具体安装请看第3点】：
下载并安装Anaconda。打开Anaconda Prompt并执行如下命令，创建名为tensorflow的虚拟环境。// 基于当前系统环境最优先的python创建tensorflow虚拟环境，例如使用Anaconda自带的python安装时，使用该语句conda create -n tensorflow// 指定python版本创建tensorflow虚拟环境，如果系统没有指定的python版本，则会下载python及其相关的插件，如numpy等conda create -n tensorflow python=3.5
--- 下文会针对这两种方式进行详细的讨论。
继续在Anaconda Prompt执行【activate tensorflow】命令，激活tensorflow环境：
下载【tensorflow_gpu-1.0.0-cp35-cp35m-win_amd64.whl】文件（下文简称【.whl】文件）并执行如下命令：
// 通过事先下载好的.whl配置tensorflowpip install --ignore-installed --upgrade tensorflow_gpu-1.0.0-cp35-cp35m-win_amd64.whl// 通过指定whl的地址配置tensorflowpip install --ignore-installed --upgrade /tensorflow/windows/gpu/tensorflow_gpu-1.0.1-cp35-cp35m-win_amd64.whl
测试tensorflow是否安装成功，在Anaconda Prompt命令窗口中依次输入如下命令：打开Anaconda集成环境下的spyder IDE进行程序开发。【可选】下载PyCharm并配置tensorflow环境。
1. 所需工具及其相关资源：
【Anaconda】（本文为Anaconda 4.3.0）：https://www.continuum.io/downloads#windows【tensorflow_gpu-1.0.0-cp35-cp35m-win_amd64.whl】：/pypi/tensorflow-gpu【PyCharm】/pycharm/download/#section=windows2. Anaconda简介：
下文从Anaconda上摘抄，此处不做翻译，原文更能体现原意。
GET SUPERPOWERS WITHANACONDA
Anaconda is the leading open data science platform powered by Python. The open source version of Anaconda is a high performance distribution of Python and R and includes over 100 of the most popular Python, R and Scala packages for data science.
Additionally, you'll have access to over 720 packages that can easily be installed with conda, our renowned package, dependency and environment manager, that is included in Anaconda. See thepackagesincluded with Anaconda and the Anacondachangelog
Which version should I download and install?
With Anaconda you can run multiple versions of Python in isolated environments, so choose the download with the Python version that you use more often, as that will be your default Python version.
If you don't have time or disk space for the entire distribution, tryMinicondawhich contains only conda and Python. Then install just the individual packages you want through the conda command.
3. tensorflow详细步骤：【读者们重点关注】
第一种方式：Anaconda的python版本=3.5，【.whl】对应的python版本=3.5--- 下载Anaconda4.2.0（对应python=3.5）
--- 条件：Anaconda对应的python版本为3.5（对应tensorflow_gpu-1.0.0-cp35-cp35m-win_amd64.wh，则以此类推）
--- 先在Anaconda Prompt下执行python，查看当前系统最优先的python版本（本人的PC上有自己安装的python2.7版本）如果在安装Anaconda之前，你的电脑上已经存在python（如果电脑上只有Anaconda提供的python就不会出现该问题），这时可能发生如下事件：
--- 打开Anaconda Prompt，执行“python”，可看到当前python的版本为3.5.2。--- 执行“activate tensorflow”激活tensorflow--- 再次执行“python”，你会发现现在的python版本改变了，变成2.7.11（这个版本是之前安装的，用于python web），个人觉得的原因是：“你执行完activate tensorflow激活后，再次执行python，此时python命令是从当前的系统环境变量path里面去寻找python.exe，正好在path中python2.7.11的路径在Anaconda之前”--- 解决办法：修改系统环境变量path的顺序，将Anaconda的路径移到python2.7.11之前。此时，可再次尝试一下如下命令，即可保持执行Anaconda的python版本。
处理好上述问题后，依次执行如下语句：
// 在Anaconda/envs目录下生成tensorflow的激活文档conda create -n tensorflow// 激活tensorflowactivate tensorflow// 配置tensorflow框架pip install --ignore-installed --upgrade /tensorflow/windows/gpu/tensorflow_gpu-1.0.1-cp35-cp35m-win_amd64.whl
此时，执行完最后一个语句的时候可能会遇到如下问题：
PermissionError: [Errno 13] Permission denied:'d:\\usr\\anaconda2\\……
-- 解决办法：重启电脑，再次运行下述语句即可解决，参考这里
pip install --upgrade --ignore-installed /tensorflow/windows/cpu/tensorflow-0.12.0rc0-cp35-cp35m-win_amd64.whl
tensorflow成功配置完成
第二种方式：Anaconda的python版本=3.6，【.whl】对应的python版本=3.5
【导读】为什么要介绍第二种方式呢？因为本人一开始在安装的过程中使用的是这种方式，遇到了些问题，再次整理归纳出来，仅供读者参考。已经通过第一种方式成功安装的读者，就可以不用看第二种方式啦。
根据本文提供的相关链接下载Anaconda，此处选择的是最新版本：Anaconda 4.3.0 for windows（64-BIT, Python 3.6 version）。由于目前python package最新提供给windows的“tensorflow-gpu”对应的python版本是3.5，虽然此处选择的Anaconda 4.3.0对应的python版本是3.6，但不影响tensorflow-gpu的版本（因为下文通过指定python版本=3.5进行配置）。tensorflow-gpu的网站如下图所示：
成功安装完Anaconda后，相关的集成工具（python等）也相应的安装完毕了。本文中Anaconda的安装过程都是根据默认设置，包括安装路径和一些选项（比如，默认勾选了安装时将路径添加到系统的环境变量中）。
打开Anaconda的Prompt，输入相关命令创建Tensorflow虚拟环境：
conda create -n tensorflow python=3.5 // 指定python版本，创建tensorflow虚拟环境
进入Tensorflow虚拟环境，并输入：activate tensorflow 激活tensorflow
-- 相关命令解释：
activate tensorflow // 激活tensorflow虚拟环境 deactivate tensorflow // 关闭tensorflow虚拟环境
激活tensorflow虚拟环境后的，可以看到输入提示符前加了（tensorflow），如下图所示：
安装【-1.0.0-cp35-cp35m-win_amd64.whl】：
tensorflow_gpu // 包含GPU模块1.0.0 // tensorflow版本cp35-cp35m //对应python的版本为3.5win_amd64 // 适应64位的windows系统.whl // 文件后缀
下载【tensorflow_gpu-1.0.0-cp35-cp35m-win_amd64.whl】文件，并将该文件放入到Anaconda Prompt命令窗口显示的目录下（因为在Prompt窗口内试图使用cd去切换目录不成功），然后输入如下命令进行安装：
pip install --ignore-installed --upgrade tensorflow_gpu-1.0.0-cp35-cp35m-win_amd64.whl
-- 效果图如下所示，下载了一大堆东西：
-- 需要注意的是，有的教程直接输入命令“pip install tensorflow”进行安装tensorflow。因为本文也参考了“极客学院”的教程，再根据一些其他搜索，找到了前文提到的“tensorflow_gpu-1.0.0-cp35-cp35m-win_amd64.whl”文件，这类文件可以区分我们安装了tensorflow是否包含GPU模块（本文安装的有包含GPU模块）
测试tensorflow是否安装成功，在Anaconda Prompt命令窗口中依次输入如下命令：
python // 进入python命令模式 import tensorflow as tf hello = tf.constant('Hello, TensorFlow!') s = tf.Session() print s.run(hello)
4. 安装并破解PyCharm：
安装PyCharm：直接在官网下载相关版本的PyCharm（/pycharm/download/#section=windows），安装过程很简单，重要的是安装成功后的激活过程，验证码到这里去找：
CNEKJPQZEX-eyJsaWNlbnNlSWQiOiJDTkVLSlBRWkVYIiwibGljZW5zZWVOYW1lIjoibGFuIHl1IiwiYXNzaWduZWVOYW1lIjoiIiwiYXNzaWduZWVFbWFpbCI6IiIsImxpY2Vuc2VSZXN0cmljdGlvbiI6IkZvciBlZHVjYXRpb25hbCB1c2Ugb25seSIsImNoZWNrQ29uY3VycmVudFVzZSI6ZmFsc2UsInByb2R1Y3RzIjpbeyJjb2RlIjoiQUMiLCJwYWlkVXBUbyI6IjIwMTgtMDEtMzAifSx7ImNvZGUiOiJETSIsInBhaWRVcFRvIjoiMjAxOC0wMS0zMCJ9LHsiY29kZSI6IklJIiwicGFpZFVwVG8iOiIyMDE4LTAxLTMwIn0seyJjb2RlIjoiUlMwIiwicGFpZFVwVG8iOiIyMDE4LTAxLTMwIn0seyJjb2RlIjoiV1MiLCJwYWlkVXBUbyI6IjIwMTgtMDEtMzAifSx7ImNvZGUiOiJEUE4iLCJwYWlkVXBUbyI6IjIwMTgtMDEtMzAifSx7ImNvZGUiOiJSQyIsInBhaWRVcFRvIjoiMjAxOC0wMS0zMCJ9LHsiY29kZSI6IlBTIiwicGFpZFVwVG8iOiIyMDE4LTAxLTMwIn0seyJjb2RlIjoiREMiLCJwYWlkVXBUbyI6IjIwMTgtMDEtMzAifSx7ImNvZGUiOiJEQiIsInBhaWRVcFRvIjoiMjAxOC0wMS0zMCJ9LHsiY29kZSI6IlJNIiwicGFpZFVwVG8iOiIyMDE4LTAxLTMwIn0seyJjb2RlIjoiUEMiLCJwYWlkVXBUbyI6IjIwMTgtMDEtMzAifSx7ImNvZGUiOiJDTCIsInBhaWRVcFRvIjoiMjAxOC0wMS0zMCJ9XSwiaGFzaCI6IjUxOTU1OTMvMCIsImdyYWNlUGVyaW9kRGF5cyI6MCwiYXV0b1Byb2xvbmdhdGVkIjpmYWxzZSwiaXNBdXRvUHJvbG9uZ2F0ZWQiOmZhbHNlfQ==-QOxwjWvRwJz6vo6J6adC3CJ4ukQHosbPYZ94URUVFna/Rbew8xK/M5gP3kAaPh6ZDveFdtMR1UBoumq3eCwXtXM3U3ls5noB4LIr+QplVlCj2pK5uNq7g/feyNyQcHpSXtvhIOnXDBLOecB05DOsxzm0p7ulGGJoAInmHeb9mc0eYjqc4RPpUQfh6HSYBnvEnKMlLF5bz4KEtzmsvvgA55CwzwQ3gRitm5Q/wUT7AQCBdjmBfNUjKVQL6TSjSDPp56FUdEs4Aab8LqstA2DIMbxocO64rvytmcUeIwu8Mi5uq87KQP5AQMSMYb59Inbd+dmVfx5cJo3fRS4/5s3/Hg==-MIIEPjCCAiagAwIBAgIBBTANBgkqhkiG9w0BAQsFADAYMRYwFAYDVQQDDA1KZXRQcm9maWxlIENBMB4XDTE1MTEwMjA4MjE0OFoXDTE4MTEwMTA4MjE0OFowETEPMA0GA1UEAwwGcHJvZDN5MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxcQkq+zdxlR2mmRYBPzGbUNdMN6OaXiXzxIWtMEkrJMO/5oUfQJbLLuMSMK0QHFmaI37WShyxZcfRCidwXjot4zmNBKnlyHodDij/78TmVqFl8nOeD5+07B8VEaIu7c3E1N+e1doC6wht4I4+IEmtsPAdoaj5WCQVQbrI8KeT8M9VcBIWX7fD0fhexfg3ZRt0xqwMcXGNp3DdJHiO0rCdU+Itv7EmtnSVq9jBG1usMSFvMowR25mju2JcPFp1+I4ZI+FqgR8gyG8oiNDyNEoAbsR3lOpI7grUYSvkB/xVy/VoklPCK2h0f0GJxFjnye8NT1PAywoyl7RmiAVRE/EKwIDAQABo4GZMIGWMAkGA1UdEwQCMAAwHQYDVR0OBBYEFGEpG9oZGcfLMGNBkY7SgHiMGgTcMEgGA1UdIwRBMD+AFKOetkhnQhI2Qb1t4Lm0oFKLl/GzoRykGjAYMRYwFAYDVQQDDA1KZXRQcm9maWxlIENBggkA0myxg7KDeeEwEwYDVR0lBAwwCgYIKwYBBQUHAwEwCwYDVR0PBAQDAgWgMA0GCSqGSIb3DQEBCwUAA4ICAQC9WZuYgQedSuOc5TOUSrRigMw4/+wuC5EtZBfvdl4HT/8vzMW/oUlIP4YCvA0XKyBaCJ2iX+ZCDKoPfiYXiaSiH+HxAPV6J79vvouxKrWg2XV6ShFtPLP+0gPdGq3x9R3+kJbmAm8w+FOdlWqAfJrLvpzMGNeDU14YGXiZ9bVzmIQbwrBA+c/F4tlK/DV07dsNExihqFoibnqDiVNTGombaU2dDup2gwKdL81ua8EIcGNExHe82kjF4zwfadHk3bQVvbfdAwxcDy4xBjs3L4raPLU3yenSzr/OEur1+jfOxnQSmEcMXKXgrAQ9U55gwjcOFKrgOxEdek/Sk1VfOjvS+nuM4eyEruFMfaZHzoQiuw4IqgGc45ohFH0UUyjYcuFxxDSU9lMCv8qdHKm+wnPRb0l9l5vXsCBDuhAGYD6ss+Ga+aDY6f/qXZuUCEUOH3QUNbbCUlviSz6+GiRnt1kA9N2Qachl+2yBfaqUqr8h7Z2gsx5LcIf5kYNsqJ0GavXTVyWh7PYiKX4bs354ZQLUwwa/cG++2+wNWP+HtBhVxMRNTdVhSm38AknZlD+PTAsWGu9GyLmhti2EnVwGybSD2Dxmhxk3IPCkhKAK+pl0eWYGZWG3tJ9mZ7SowcXLWDFAk0lRJnKGFMTggrWjV8GYpw5bq23VmIqqDLgkNzuoog==
在PyCharm下配置tensorflow环境：
要在pycharm下使用tensorflow，要设置好pycharm下解释器interpreter的路径【File --> Setting --> Project --> Project Interpreter】，这里也就是tensorflow的路径，如下图所示：
基于PyCharm的案例测试：
import tensorflow as tf hello = tf.constant('Hello, TensorFlow!') sess = tf.Session() print (sess.run(hello)) a = tf.constant(10) b = tf.constant(32) print (sess.run(a+b))
-- 执行结果如下所示：其中console会输出一些警告，使用第一种方式就不会。
5. tensorflow的相关案例
官网上tensorflow的基本用法：import tensorflow as tfimport numpy as npdef linear_test():# Model parametersprint("linear_test")W = tf.Variable([.3], tf.float32)b = tf.Variable([-.3], tf.float32)# Model input and outputx = tf.placeholder(tf.float32)linear_model = W * x + by = tf.placeholder(tf.float32)# lossloss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares# optimizeroptimizer = tf.train.GradientDescentOptimizer(0.01)train = optimizer.minimize(loss)# training datax_train = [1, 2, 3, 4]y_train = [0, -1, -2, -3]# training loopinit = tf.global_variables_initializer()sess = tf.Session()sess.run(init) # reset values to wrongfor i in range(1000):sess.run(train, {x: x_train, y: y_train})# evaluate training accuracycurr_W, curr_b, curr_loss = sess.run([W, b, loss], {x: x_train, y: y_train})print("W: %s b: %s loss: %s" % (curr_W, curr_b, curr_loss))def contrib_test():print("contrib_test")# Declare list of features. We only have one real-valued feature. There are many# other types of columns that are more complicated and useful.features = [tf.contrib.layers.real_valued_column("x", dimension=1)]# An estimator is the front end to invoke training (fitting) and evaluation# (inference). There are many predefined types like linear regression,# logistic regression, linear classification, logistic classification, and# many neural network classifiers and regressors. The following code# provides an estimator that does linear regression.estimator = tf.contrib.learn.LinearRegressor(feature_columns=features)# TensorFlow provides many helper methods to read and set up data sets.# Here we use `numpy_input_fn`. We have to tell the function how many batches# of data (num_epochs) we want and how big each batch should be.x = np.array([1., 2., 3., 4.])y = np.array([0., -1., -2., -3.])input_fn = tf.contrib.learn.io.numpy_input_fn({"x": x}, y, batch_size=4, num_epochs=1000)# We can invoke 1000 training steps by invoking the `fit` method and passing the# training data set.estimator.fit(input_fn=input_fn, steps=1000)# Here we evaluate how well our model did. In a real example, we would want# to use a separate validation and testing data set to avoid overfitting.print(estimator.evaluate(input_fn=input_fn))# Declare list of features, we only have one real-valued featuredef model(features, labels, mode):# Build a linear model and predict valuesW = tf.get_variable("W", [1], dtype=tf.float64)b = tf.get_variable("b", [1], dtype=tf.float64)y = W*features['x'] + b# Loss sub-graphloss = tf.reduce_sum(tf.square(y - labels))# Training sub-graphglobal_step = tf.train.get_global_step()optimizer = tf.train.GradientDescentOptimizer(0.01)train = tf.group(optimizer.minimize(loss), tf.assign_add(global_step, 1))# ModelFnOps connects subgraphs we built to the# appropriate functionality.return tf.contrib.learn.ModelFnOps(mode=mode, predictions=y,loss=loss,train_op=train)def custom_model():print("custom_model")estimator = tf.contrib.learn.Estimator(model_fn=model)# define our data setx = np.array([1., 2., 3., 4.])y = np.array([0., -1., -2., -3.])input_fn = tf.contrib.learn.io.numpy_input_fn({"x": x}, y, 4, num_epochs=1000)# trainestimator.fit(input_fn=input_fn, steps=1000)# evaluate our modelprint(estimator.evaluate(input_fn=input_fn, steps=10))def linear2():print("linear2")# 使用 NumPy 生成假数据(phony data), 总共 100 个点.x_data = np.float32(np.random.rand(2, 100)) # 随机输入y_data = np.dot([0.100, 0.200], x_data) + 0.300# 构造一个线性模型#b = tf.Variable(tf.zeros([1]))W = tf.Variable(tf.random_uniform([1, 2], -1.0, 1.0))y = tf.matmul(W, x_data) + b# 最小化方差loss = tf.reduce_mean(tf.square(y - y_data))optimizer = tf.train.GradientDescentOptimizer(0.5)train = optimizer.minimize(loss)# 初始化变量init = tf.global_variables_initializer()# 启动图 (graph)#with tf.Session() as sess:# with tf.device("/gpu:1"):# matrix1 = tf.constant([[3., 3.]])# matrix2 = tf.constant([[2.], [2.]])# product = tf.matmul(matrix1, matrix2)sess = tf.Session()sess.run(init)# 拟合平面for step in range(0, 201):sess.run(train)if step % 20 == 0:print(step, sess.run(W), sess.run(b))if __name__ == '__main__':linear_test()contrib_test()custom_model()linear2()
-- 执行结果如下（console处的输出信息）：
linear_testW: [-0.9999969] b: [ 0.99999082] loss: 5.69997e-11contrib_test{'loss': 2.3632133e-06, 'global_step': 1000}custom_model{'loss': 1.5870483e-10, 'global_step': 1000}linear20 [[ 1.10642493 -0.51869768]] [ 0.36463627]20 [[ 0.28492442 0.07607564]] [ 0.26973405]40 [[ 0.1388548 0.18315867]] [ 0.28858268]60 [[ 0.10885192 0.19865625]] [ 0.29602784]80 [[ 0.10220318 0.20028883]] [ 0.29866859]100 [[ 0.10059496 0.20022063]] [ 0.29956201]120 [[ 0.10017135 0.20009363]] [ 0.29985732]140 [[ 0.1000516 0.20003414]] [ 0.29995376]160 [[ 0.10001599 0.2000117 ]] [ 0.29998505]180 [[ 0.10000505 0.20000389]] [ 0.29999518]200 [[ 0.10000161 0.20000128]] [ 0.29999843]
MNIST：
未引入CNN的MNIST：
import tensorflow as tfdef mnist_orical():print('begin example for mnist') #/news/02/vJpJqREn7EyoAd09.htmlfrom tensorflow.examples.tutorials.mnist import input_datatf.reset_default_graph()mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)x = tf.placeholder(tf.float32, [None, 784]) # 行表示样本，列表示值y_ = tf.placeholder(tf.float32, [None, 10]) # 行表示样本，列表示值W = tf.Variable(tf.zeros([784, 10]), name='weights')b = tf.Variable(tf.zeros([10]), name='biases')y = tf.nn.softmax(tf.matmul(x, W) + b) # softmaxcross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), 1))# 1表示对每一行（的所有列）操作，0表示对每一列（的所有行）操作，这里y_*tf.log(y)是对应元素运算？# 以下定义学习步长0.5，使用了梯度下降法train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)init = tf.global_variables_initializer()# 以上是graph定义完毕# 下面通过session进行真正的计算print('begin optimization')with tf.Session() as sess:sess.run(init)for i in range(2000):batch_xs, batch_ys = mnist.train.next_batch(100)# sess.run(train_step, feed_dict={x: batch_xs, y_:batch_ys})_, loss_val = sess.run([train_step, cross_entropy], feed_dict={x: batch_xs, y_: batch_ys})# fetch the value of ** by adding it to the list of arguments to sess.run(...)print('iter: %d , loss = %s' % (i, loss_val))# 迭代1000次，每次输入100个样本。mnist.train.next_batch 就是生成下一个 batch 的数据correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))print('begin test')# a6 = mnist.test.images[0]#[1000,784],每行是一个784维的样本，归一化到0-1# print((a6))print(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))print('test done')summary_writer = tf.summary.FileWriter('mnist_logs', sess.graph) ##???tf.reset_default_graph()if __name__ == '__main__':# mnist_cnn()mnist_orical()
-- 执行结果如下：
引入CNN的MNIST：
from tensorflow.examples.tutorials.mnist import input_datamnist = input_data.read_data_sets('MNIST_data', one_hot=True)import tensorflow as tfsess = tf.InteractiveSession()x = tf.placeholder(tf.float32, shape=[None, 784])y_ = tf.placeholder(tf.float32, shape=[None, 10])W = tf.Variable(tf.zeros([784, 10]))b = tf.Variable(tf.zeros([10]))y = tf.nn.softmax(tf.matmul(x, W) + b)def weight_variable(shape):initial = tf.truncated_normal(shape, stddev=0.1)return tf.Variable(initial)def bias_variable(shape):initial = tf.constant(0.1, shape=shape)return tf.Variable(initial)def conv2d(x, W):return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')def max_pool_2x2(x):return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],strides=[1, 2, 2, 1], padding='SAME')W_conv1 = weight_variable([5, 5, 1, 32])b_conv1 = bias_variable([32])x_image = tf.reshape(x, [-1,28,28,1])h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)h_pool1 = max_pool_2x2(h_conv1)W_conv2 = weight_variable([5, 5, 32, 64])b_conv2 = bias_variable([64])h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)h_pool2 = max_pool_2x2(h_conv2)W_fc1 = weight_variable([7 * 7 * 64, 1024])b_fc1 = bias_variable([1024])h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)keep_prob = tf.placeholder(tf.float32)h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)W_fc2 = weight_variable([1024, 10])b_fc2 = bias_variable([10])y_conv=tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv), reduction_indices=[1]))train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(y_,1))accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))init = tf.global_variables_initializer()config = tf.ConfigProto()config.gpu_options.allocator_type = 'BFC'with tf.Session(config = config) as s:sess.run(init)for i in range(2000):batch = mnist.train.next_batch(50)if i%100 == 0:train_accuracy = accuracy.eval(feed_dict={x:batch[0], y_: batch[1], keep_prob: 1.0})print("step %d, training accuracy %g"%(i, train_accuracy))train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})# print("test accuracy %g"%accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))print("test accuracy %g" % accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))
还原MNIST图像：参考这里
#!/usr/bin/env python# -*- coding: utf-8 -*-from PIL import Imageimport structdef read_image(filename):f = open(filename, 'rb')index = 0buf = f.read()f.close()magic, images, rows, columns = struct.unpack_from('>IIII' , buf , index)index += struct.calcsize('>IIII')# xrange has become range in python 3for i in range(images):# for i in xrange(2000):# 创建一张空白的图片，其中的’L’代表这张图片是灰度图image = Image.new('L', (columns, rows))for x in range(rows):for y in range(columns):image.putpixel((y, x), int(struct.unpack_from('>B', buf, index)[0]))index += struct.calcsize('>B')print('save ' + str(i) + ' image')image.save('test_data/' + str(i) + '.png')def read_label(filename, saveFilename):f = open(filename, 'rb')index = 0buf = f.read()f.close()magic, labels = struct.unpack_from('>II' , buf , index)index += struct.calcsize('>II')labelArr = [0] * labels#labelArr = [0] * 2000for x in range(labels):#for x in xrange(2000):labelArr[x] = int(struct.unpack_from('>B', buf, index)[0])index += struct.calcsize('>B')save = open(saveFilename, 'w')save.write(','.join(map(lambda x: str(x), labelArr)))save.write('\n')save.close()print('save labels success')if __name__ == '__main__':read_image('t10k-images.idx3-ubyte')read_label('t10k-labels.idx1-ubyte', 'test_data/label.txt')
还原后的MNIST图像如下：MNIST数据结构解析参考这里
6. 遇到的问题及其解决方案：
问题1：如下所示一些警告。使用安装方式二，即使用【conda create -n tensorflow python=3.5】方法，会产生如下警告。使用方式一，即基于Anaconda自带的python进行配置tensorflow环境，就不会出现类似的警告。E c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\:943] OpKernel ('op: "BestSplits" device_type: "CPU"') for unknown op: BestSplitsE c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\:943] OpKernel ('op: "CountExtremelyRandomStats" device_type: "CPU"') for unknown op: CountExtremelyRandomStatsE c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\:943] OpKernel ('op: "FinishedNodes" device_type: "CPU"') for unknown op: FinishedNodesE c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\:943] OpKernel ('op: "GrowTree" device_type: "CPU"') for unknown op: GrowTreeE c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\:943] OpKernel ('op: "ReinterpretStringToFloat" device_type: "CPU"') for unknown op: ReinterpretStringToFloatE c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\:943] OpKernel ('op: "SampleInputs" device_type: "CPU"') for unknown op: SampleInputsE c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\:943] OpKernel ('op: "ScatterAddNdim" device_type: "CPU"') for unknown op: ScatterAddNdimE c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\:943] OpKernel ('op: "TopNInsert" device_type: "CPU"') for unknown op: TopNInsertE c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\:943] OpKernel ('op: "TopNRemove" device_type: "CPU"') for unknown op: TopNRemoveE c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\:943] OpKernel ('op: "TreePredictions" device_type: "CPU"') for unknown op: TreePredictionsE c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\core\framework\:943] OpKernel ('op: "UpdateFertileSlots" device_type: "CPU"') for unknown op: UpdateFertileSlots
问题2：GPU显存不够。Tensorflow Deep MNIST: Resource exhausted: OOM when allocating tensor with shape[10000,32,28,28]，参考这里。
