#原文鏈接：https://medium.com/deep-learning-turkey/google-colab-free-gpu-tutorial-e113627b9f5d
Now you can develop deep learning applications with Google Colaboratory -on the free Tesla K80 GPU- using Keras, Tensorflow and PyTorch.
Hello! I will show you how to use Google Colab, Google’s free cloud service for AI developers. With Colab, you can develop deep learning applications on the GPU for free.
Thanks to KDnuggets!
I am happy to announce that this blog post was selected as KDnuggets Silver Blog for February 2018! Read this on KDnuggets.
What is Google Colab?
Google Colab is a free cloud service and now it supports free GPU!
You can;
improve your Python programming language coding skills.
develop deep learning applications using popular libraries such as Keras, TensorFlow, PyTorch, and OpenCV.
The most important feature that distinguishes Colab from other free cloud services is; Colab provides GPU and is totally free.
Detailed information about the service can be found on the faq page.
Getting Google Colab Ready to Use
Creating Folder on Google Drive
Since Colab is working on your own Google Drive, we first need to specify the folder we’ll work. I created a folder named “app” on my Google Drive. Of course, you can use a different name or choose the default Colab Notebooks folder instead of app folder.
I created an empty “app” folder
Creating New Colab Notebook
Create a new notebook via Right click > More > Colaboratory
Right click > More > Colaboratory
Rename notebook by means of clicking the file name.
Setting Free GPU
It is so simple to alter default hardware (CPU to GPU or vice versa); just follow Edit > Notebook settings or Runtime>Change runtime type and select GPU as Hardware accelerator.
Running Basic Python Codes with Google Colab
Now we can start using Google Colab.
I will run some Basic Data Types codes from Python Numpy Tutorial.
It works as expected :) If you do not know Python which is the most popular programming language for AI, I would recommend this simple and clean tutorial.
Running or Importing .py Files with Google Colab
Run these codes first in order to install the necessary libraries and perform authorization.
When you run the code above, you should see a result like this:
Click the link, copy verification code and paste it to text box.
After completion of the authorization process, you should see this:
Now you can reach you Google Drive with:
install Keras:
!pip install -q keras
upload mnist_cnn.py file to app folder which is located on your Google Drive.
mnist_cnn.py file
run the code below to train a simple convnet on the MNIST dataset.
!python3 "/content/drive/My Drive/app/mnist_cnn.py"
As you can see from the results, each epoch lasts only 11 seconds.
Download Titanic Dataset (.csv File) and Display First 5 Rows
If you want to download.csv file from url to “app” folder, simply run:
!wgethttps://raw.githubusercontent.com/vincentarelbundock/Rdatasets/master/csv/datasets/Titanic.csv-P "/content/drive/My Drive/app"
You may upload your .csv files directly to “app” folder instead of wget method.
Read.csv file in “app” folder and display first 5 rows:
import pandas as pd
titanic = pd.read_csv(“/content/drive/My Drive/app/Titanic.csv”)
titanic.head(5)
Cloning Github Repo to Google Colab
It is easy to clone a Github repo with Git.
Step 1: Find the Github Repo and Get “Git” Link
Find any Github repo to use.
For instance: https://github.com/wxs/keras-mnist-tutorial
Clone or download > Copy the link!
2. Git Clone
Simply run:
!git clonehttps://github.com/wxs/keras-mnist-tutorial.git
3. Open the Folder in Google Drive
Folder has the same with the Github repo of course :)
4. Open The Notebook
Right Click > Open With > Colaboratory
5. Run
Now you are able to run Github repo in Google Colab.
Some Useful Tips
1. How to Install Libraries?
Keras
!pip install -q keras
import keras
PyTorch
from os import path
from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag
platform = '{}{}-{}'.format(get_abbr_impl(), get_impl_ver(), get_abi_tag())
accelerator = 'cu80' if path.exists('/opt/bin/nvidia-smi') else 'cpu'
!pip install -qhttp://download.pytorch.org/whl/{accelerator}/torch-0.3.0.post4-{platform}-linux_x86_64.whltorchvision
import torch
or try this:
!pip3 install torch torchvision
MxNet
!apt install libnvrtc8.0
!pip install mxnet-cu80
import mxnet as mx
OpenCV
!apt-get -qq install -y libsm6 libxext6 && pip install -q -U opencv-python
import cv2
XGBoost
!pip install -q xgboost==0.4a30
import xgboost
GraphViz
!apt-get -qq install -y graphviz && pip install -q pydot
import pydot
7zip Reader
!apt-get -qq install -y libarchive-dev && pip install -q -U libarchive
import libarchive
Other Libraries
!pip install or!apt-get install to install other libraries.
2. Is GPU Working?
To see if you are currently using the GPU in Colab, you can run the following code in order to cross-check:
import tensorflow as tf
tf.test.gpu_device_name()
3. Which GPU Am I Using?
from tensorflow.python.client import device_lib
device_lib.list_local_devices()
Currently, Colab only provides Tesla K80.
4. What about RAM?
!cat /proc/meminfo
5. What about CPU?
!cat /proc/cpuinfo
6. Changing Working Directory
Normally when you run this code:
!ls
You probably see datalab and drive folders.
Therefore you must add drive/app before defining each filename.
To get rid of this problem, you can simply change the working directory. (In this tutorial I changed to app folder) with this simple code:
import os
os.chdir("drive/app")
After running code above, if you run again
!ls
You would see app folder content and don’t need to add drive/app all the time anymore.
7. “No backend with GPU available“ Error Solution
If you encounter this error:
Failed to assign a backend
No backend with GPU available. Would you like to use a runtime with no accelerator?
Try again a bit later. A lot of people are kicking the tires on GPUs right now, and this message arises when all GPUs are in use.
Reference
8. How to Clear Outputs of All Cells
Follow Tools>>Command Palette>>Clear All Outputs
9. “apt-key output should not be parsed (stdout is not a terminal)” Warning
If you encounter this warning:
Warning: apt-key output should not be parsed (stdout is not a terminal)
That means authentication has already done. You only need to mount Google Drive:
!mkdir -p drive
!google-drive-ocamlfuse drive
10. How to Use Tensorboard with Google Colab?
I recommend this repo:
https://github.com/mixuala/colab_utils
11. How to Restart Google Colab?
In order to restart (or reset) your virtual machine, simply run:
!kill -9 -1
12. How to Add Form to Google Colab?
In order not to change hyperparameters every time in your code, you can simply add form to Google Colab.
For instance, I added form which contain learning_rate variable and optimizer string.
13. How to See Function Arguments?
To see function arguments in TensorFlow, Keras etc, simply add question mark (?) after function name:
Now you can see original documentation without clicking TensorFlow website.
14. How to Send Large Files From Colab To Google Drive?
15. How to Run Tensorboard in Google Colab?
If you want to runt Tensorboard in Google Colab, run the code below.
You can track your Tensorboard logs with created ngrok.io URL. You will find the URL at the end of output.
Note that your Tensorboard logs will be save to tb_logs dir. Of course, you can change the directory name.
After that, we can see the Tensorboard in action! After running the code below, you can track you Tensorboard logs via ngrok URL.
Tensorboard :)
Conclusion
I think Colab will bring a new breath to Deep Learning and AI studies all over the world.
If you found this article helpful, it would mean a lot if you gave it some applause and shared to help others find it! And feel free to leave a comment below.
You can find me on Twitter.
Last Note
This blog post will be constantly updated.
Changelog
26-01–2018
“insert app folder to path” removed
“downloading, reading and displaying .csv file” added
“Some Useful Tips” added
27–01–2018
“Changing Working Directory” added
28–01–2018
“Cloning Github Repo to Google Colab” added
“pip install mxnet” added
29–01–2018
No backend with GPU available. Error Solution added
2–02–2018
“MxNet Installation” changed (CPU to GPU)
5–02–2018
“How to Clear Outputs of All Cells” added
apt-key output should not be parsed (stdout is not a terminal)warning added
11–02–2018
“How to use Tensorboard with Google Colab” added
20–02–2018
KDnuggets re-posted this tutorial.
28–02–2018
“How to Restart Google Colab?” added
9–03–2018
How to Add Form to Google Colab? added
21–03–2018
How to See Function Arguments? added
20–05–2018
PyTorch installation updated
18–08–2018
How to Send Large Files From Colab to Google Drive added
19–08–2018
How to Run Tensorboard in Google Colab added
28–09–2018
Auth and Google Drive mounting processs changed