windows安装superset并使用postgresql和redis

windows和linux安装superset

Linux篇

1. install python3

   sudo yum install -y python3 python3-devel python3-pip libpq-dev
   sudo yum install -y gcc gcc-c++
   echo "export PATH=/usr/local/bin:$PATH" >> ~/.bashrc
   echo "alias python=python3" >> ~/.bashrc
   echo "alias pip=pip3" >> ~/.bashrc
   source ~/.bashrc
   python --version
   

2. install virtualenv

mkdir /opt/superset 
cd /opt/superset
pip install virtualenv -i https://mirrors.aliyun.com/pypi/simple
virtualenv venv
. venv/bin/activate

3. install superset

wget https://raw.githubusercontent.com/apache/incubator-superset/master/requirements.txt
wget https://raw.githubusercontent.com/apache/incubator-superset/master/requirements-dev.txt
pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple
pip install -r requirements-dev.txt -i https://mirrors.aliyun.com/pypi/simple
pip install superset

5. use postgresql

install postgresql client

sudo yum install -y postgresql

connect to remote database and setup user and database

create database superset encoding='utf8';
create user superset with password 'superset';
grant all privileges on database superset to superset;

create a config file and put it to /opt/superset/venv/lib/python3.7/site-packages

SQLALCHEMY_DATABASE_URI = 'postgresql+psycopg2://superset:superset@localhost/superset'

6. config

export FLASK_APP=superset
flask fab create-admin
superset db upgrade
superset init

7. use gunicorn

gunicorn \
        --bind  "0.0.0.0:8443" \
        --workers 2 \
        --worker-class gthread \
        --threads 20 \
        --timeout 60 \
        --limit-request-line 0 \
        --limit-request-field_size 0 \
        "superset.app:create_app()"

8. redis

install redis

sudo yum install redis

9. 添加内容到superset_config.py

CACHE_DEFAULT_TIMEOUT = 86400
CACHE_CONFIG : CacheConfig= {
'CACHE_TYPE': 'redis',
'CACHE_DEFAULT_TIMEOUT': 86400,
'CACHE_KEY_PREFIX': 'superset_',
'CACHE_REDIS_HOST': 'localhost',
'CACHE_REDIS_PORT': 6379,
'CACHE_REDIS_DB': 1,
'CACHE_REDIS_URL': 'redis://localhost:6379/1'
}

查看redis缓存

redis -n 1
keys *

安装python 环境

1. 安装python 3.x, 这个不用说了

2. 安装virtualenv, 虚拟一个环境出来

   mkdir superset
   cd superset
   virtualenv venv
   venv\Scripts\activate.bat
   

安装superset

1. 先升一下pip和setup tool

   pip install --upgrade setuptools pip
   

2. 到 https://www.lfd.uci.edu/~gohlke/pythonlibs 去下个python_geohash的wheel包, 然后本地安装

   pip install python_geohash-0.8.5-cp37-cp37m-win_amd64.whl
   

3. 然后开始安装superset和它的依赖包

   pip install superset
   

   有时候会只安装一个superset, 它的依赖包不自动安装, 这时候就手动装一下

   先下载 https://github.com/apache/incubator-superset/blob/master/requirements.txt, 然后pip 安装

   pip install -r requirements.txt
   

配置

1. 建个用户

   set FLASK_APP=superset
   flask fab create-admin
   

2. 配置db

   python venv\Scripts\superset db upgrade
   

3. 初始化环境

   python venv\Scripts\superset init
   

4. 做完以上步骤就可以启动了

   python venv\Scripts\superset run -p 8080
   

   浏览器打开127.0.0.1:8080, 出现登录界面, 用flask fab那一步设置的用户登录

一些高级设置

用postgresql做数据库

• 安装win版的postgresql数据库, 建一个superset的用户, 密码也设成superset, 再建一个数据库叫superset, 把superset用户做成owner

• 建一个superset_config.py的文件, 放到venv\Libs\site-packages目录下面, 内容如下

  # The SQLAlchemy connection string.
  SQLALCHEMY_DATABASE_URI = 'postgresql+psycopg2://superset:superset@localhost/superset'
  

• 重新走一遍配置步骤

    set FLASK_APP=superset
    flask fab create-admin
    python venv\Scripts\superset db upgrade
    python venv\Scripts\superset init
  

  做db upgrade那一步的时候, 留意一下输出信息里是不是用一行

  INFO [alembic.runtime.migration] Context impl PostgresqlImpl.

  看到PostgresqlImpl.说明就成功了. 再开启usperset, 登录后在database下面就可以看到database的backend是postgresql了

用redis做cache

1. 安装 redis-server和pip redis
2. 在superset_config.py 中添加如下内容

CACHE_DEFAULT_TIMEOUT = 86400
CACHE_CONFIG : CacheConfig= {
'CACHE_TYPE': 'redis',
'CACHE_DEFAULT_TIMEOUT': 86400,
'CACHE_KEY_PREFIX': 'superset_',
'CACHE_REDIS_HOST': 'localhost',
'CACHE_REDIS_PORT': 6379,
'CACHE_REDIS_DB': 1,
'CACHE_REDIS_URL': 'redis://localhost:6379/1'
}

3. 重新启动superset

gunicorn \
        --bind  "0.0.0.0:8443" \
        --workers 2 \
        --worker-class gthread \
        --threads 20 \
        --timeout 60 \
        --limit-request-line 0 \
        --limit-request-field_size 0 \
        "superset.app:create_app()"

观察日志, 会看到有set cache key和load from cache的内容
4. 在redis中查看 key
启动redis-cli -n 1, 注意这里的-n参数对应superset_config.py中cache_redis_db的值, 用keys命令可以看到key