欢迎关注我的CSDN:https://spike.blog.csdn.net/
本文地址:https://blog.csdn.net/caroline_wendy/article/details/131231501
Kubeflow 的 Notebook 功能是一种方便的方式,让用户可以在 Kubernete s集群上创建和管理交互式的 Jupyter Notebook。用户可以通过 Kubeflow Dashboard 或者 kubectl 命令来创建和管理 Notebook 服务器。Notebook 服务器是一个 Kubernetes Deployment,运行了一个单独的 Notebook 实例,并且可以配置不同的资源和环境。用户可以在 Notebook中运行各种数据科学和机器学习的代码,以及使用 Kubeflow 提供的其他组件和服务。
配置 .bashrc
,命令如下:
cp files/.bashrc ~/.bashrc
source ~/.bashrc
具体的推荐
.bashrc
,请参考最后部分。
配置 conda 环境,安装 Miniconda,其它版本 Miniconda 或者 Anaconda 均可,命令如下:
wget https://repo.anaconda.com/miniconda/Miniconda3-py38_23.3.1-0-Linux-x86_64.sh
bash files/Miniconda3-py38_23.3.1-0-Linux-x86_64.sh
source ~/.bashrc
实际下载地址:https://repo.anaconda.com/miniconda/,约98.8M
推荐使用 Python3.8 版本
配置 Conda 源,即 .condarc
:
cp files/.condarc ~/.condarc
具体的推荐
.condarc
,请参考最后部分。
配置 pip 源,即 .pip/
:
mkdir ~/.pip
vim ~/.pip/pip.conf
# 文件内容如下:
# This file has been autogenerated or modified by NVIDIA PyIndex.
# In case you need to modify your PIP configuration, please be aware that
# some configuration files may have a priority order. Here are the following
# files that may exists in your machine by order of priority:
#
# [Priority 1] Site level configuration files
# 1. `/opt/conda/pip.conf`
#
# [Priority 2] User level configuration files
# 1. `/root/.config/pip/pip.conf`
# 2. `/root/.pip/pip.conf`
#
# [Priority 3] Global level configuration files
# 1. `/etc/pip.conf`
# 2. `/etc/xdg/pip/pip.conf`
[global]
no-cache-dir = true
index-url = https://pypi.tuna.tsinghua.edu.cn/simple/
extra-index-url = https://pypi.ngc.nvidia.com
trusted-host = pypi.tuna.tsinghua.edu.cn pypi.ngc.nvidia.com
创建 conda 环境:
conda create -n torch-def # 创建空环境
# 或者,复制已有环境,时间较长,建议慢慢等待
conda create -n torch-def --clone conda_envs/torch-def/
测试 PyTorch 环境:
import torch
# 测试环境
ngpu= 1
# Decide which device we want to run on
device = torch.device("cuda:0" if (torch.cuda.is_available() and ngpu > 0) else "cpu")
print("驱动为:",device)
print("GPU型号: ",torch.cuda.get_device_name(0))
# 测试张量
import torch
import time
print(torch.__version__)
print(torch.cuda.is_available())
a = torch.randn(100, 10)
b = torch.randn(10, 200)
device = torch.device('cuda')
a = a.to(device)
b = b.to(device)
c = torch.matmul(a, b)
print(a.device, c.device)
Kubeflow 创建 Notebook:
在 New notebook 页面,依次修改 Name、Image、CPU&Memory、GPU型号:
其余,选择默认,点击 LAUNCH 即可。
推荐的.bashrc
,如下:
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
# If not running interactively, don't do anything
case $- in
*i*) ;;
*) return;;
esac
# don't put duplicate lines or lines starting with space in the history.
# See bash(1) for more options
HISTCONTROL=ignoreboth
# append to the history file, don't overwrite it
shopt -s histappend
# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=1000
HISTFILESIZE=2000
# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize
# If set, the pattern "**" used in a pathname expansion context will
# match all files and zero or more directories and subdirectories.
#shopt -s globstar
# make less more friendly for non-text input files, see lesspipe(1)
[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)"
# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "${debian_chroot:-}" ] && [ -r /etc/debian_chroot ]; then
debian_chroot=$(cat /etc/debian_chroot)
fi
# set a fancy prompt (non-color, unless we know we "want" color)
case "$TERM" in
xterm-color|*-256color) color_prompt=yes;;
esac
# uncomment for a colored prompt, if the terminal has the capability; turned
# off by default to not distract the user: the focus in a terminal window
# should be on the output of commands, not on the prompt
#force_color_prompt=yes
if [ -n "$force_color_prompt" ]; then
if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then
# We have color support; assume it's compliant with Ecma-48
# (ISO/IEC-6429). (Lack of such support is extremely rare, and such
# a case would tend to support setf rather than setaf.)
color_prompt=yes
else
color_prompt=
fi
fi
if [ "$color_prompt" = yes ]; then
PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
else
PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi
unset color_prompt force_color_prompt
# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*)
PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"
;;
*)
;;
esac
# enable color support of ls and also add handy aliases
if [ -x /usr/bin/dircolors ]; then
test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
alias ls='ls --color=auto'
#alias dir='dir --color=auto'
#alias vdir='vdir --color=auto'
alias grep='grep --color=auto'
alias fgrep='fgrep --color=auto'
alias egrep='egrep --color=auto'
fi
# colored GCC warnings and errors
#export GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01'
# some more ls aliases
alias ll='ls -alF'
alias la='ls -A'
alias l='ls -CF'
# Add an "alert" alias for long running commands. Use like so:
# sleep 10; alert
alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"'
# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.
if [ -f ~/.bash_aliases ]; then
. ~/.bash_aliases
fi
# enable programmable completion features (you don't need to enable
# this, if it's already enabled in /etc/bash.bashrc and /etc/profile
# sources /etc/bash.bashrc).
if ! shopt -oq posix; then
if [ -f /usr/share/bash-completion/bash_completion ]; then
. /usr/share/bash-completion/bash_completion
elif [ -f /etc/bash_completion ]; then
. /etc/bash_completion
fi
fi
# my configs
export PATH=$PATH:~/.local/bin
export TORCH_HOME=workspace/torch_home/
# bos
alias bos='bcecmd --conf-path /etc/bceconf/ bos'
推荐的.condarc
,参考:
channels:
- defaults
show_channel_urls: true
default_channels:
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:
conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
msys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
bioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
menpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
pytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
simpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
channel_priority: disabled
allow_conda_downgrades: true