李春田
李春田

DataHub 0.8.14.1安装方法

好像跟之前我写的那个在安装上有区别

修改pip 源
建议用阿里云的源，豆瓣源有问题

python3 -m pip config set global.index-url  https://mirrors.aliyun.com/pypi/simple/
Writing to /root/.config/pip/pip.conf

或者

python3 -m pip install -i http://pypi.douban.com/simple --trusted-host pypi.douban.com

参考官方文档

python3 -m pip install --upgrade pip wheel setuptools
python3 -m pip uninstall datahub acryl-datahub || true  # sanity check - ok if it fails
python3 -m pip install --upgrade acryl-datahubdatahub version

插件检测及安装

 python3 -m datahub check plugins
Sources:
athena         (disabled)
azure-ad
bigquery       (disabled)
bigquery-usage (disabled)
datahub-business-glossary
dbt
druid          (disabled)
feast
file
glue           (disabled)
hive           (disabled)
kafka          (disabled)
kafka-connect  (disabled)
ldap           (disabled)
looker         (disabled)
lookml         (disabled)
mongodb        (disabled)
mssql          (disabled)
mysql          (disabled)
okta           (disabled)
oracle         (disabled)
postgres       (disabled)
redash         (disabled)
redshift       (disabled)
sagemaker      (disabled)
snowflake      (disabled)
snowflake-usage(disabled)
sqlalchemy     (disabled)
superset

Sinks:
console
datahub-kafka  (disabled)
datahub-rest
file

Transformers:
add_dataset_ownership
add_dataset_tags
mark_dataset_status
pattern_add_dataset_ownership
set_dataset_browse_path
simple_add_dataset_ownership
simple_add_dataset_tags
simple_remove_dataset_ownership

For details on why a plugin is disabled, rerun with '--verbose'
If a plugin is disabled, try running: pip install 'acryl-datahub[]'

根据官方文档安装mysql及postgresql插件

python3 -m pip install 'acryl-datahub[postgres]'
python3 -m pip install 'acryl-datahub[mysql]'

postgresql
python3 -m pip install 'acryl-datahub[postgres]'
Requirement already satisfied: acryl-datahub[postgres] in /usr/local/python3/lib/python3.8/site-packages (0.8.14.1)
Requirement already satisfied: typing-inspect in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (0.7.1)
Requirement already satisfied: entrypoints in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (0.3)
Requirement already satisfied: toml>=0.10.0 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (0.10.2)
Requirement already satisfied: tabulate in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (0.8.9)
Requirement already satisfied: mypy-extensions>=0.4.3 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (0.4.3)
Requirement already satisfied: expandvars>=0.6.5 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (0.7.0)
Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (2.8.2)
Requirement already satisfied: avro-python3>=1.8.2 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (1.10.1)
Requirement already satisfied: pydantic>=1.5.1 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (1.8.2)
Requirement already satisfied: click>=6.0.0 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (8.0.1)
Requirement already satisfied: click-default-group in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (1.2.2)
Requirement already satisfied: stackprinter in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (0.2.5)
Requirement already satisfied: docker in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (4.4.1)
Requirement already satisfied: PyYAML in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (5.3.1)
Requirement already satisfied: avro-gen3==0.6.0 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[postgres]) (0.6.0)
Collecting sqlalchemy==1.3.24
  Downloading SQLAlchemy-1.3.24-cp38-cp38-manylinux2010_x86_64.whl (1.3 MB)
     |████████████████████████████████| 1.3 MB 194 kB/s 
Collecting GeoAlchemy2
  Downloading GeoAlchemy2-0.9.4-py2.py3-none-any.whl (32 kB)
Collecting psycopg2-binary
  Downloading psycopg2_binary-2.9.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)
     |████████████████████████████████| 3.4 MB 3.8 MB/s 
Requirement already satisfied: pytz in /usr/local/python3/lib/python3.8/site-packages (from avro-gen3==0.6.0->acryl-datahub[postgres]) (2021.1)
Requirement already satisfied: tzlocal in /usr/local/python3/lib/python3.8/site-packages (from avro-gen3==0.6.0->acryl-datahub[postgres]) (3.0)
Requirement already satisfied: six in /usr/local/python3/lib/python3.8/site-packages (from avro-gen3==0.6.0->acryl-datahub[postgres]) (1.15.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/python3/lib/python3.8/site-packages (from pydantic>=1.5.1->acryl-datahub[postgres]) (3.10.0.2)
Requirement already satisfied: websocket-client>=0.32.0 in /usr/local/python3/lib/python3.8/site-packages (from docker->acryl-datahub[postgres]) (0.57.0)
Requirement already satisfied: requests!=2.18.0,>=2.14.2 in /usr/local/python3/lib/python3.8/site-packages (from docker->acryl-datahub[postgres]) (2.25.1)
Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/python3/lib/python3.8/site-packages (from requests!=2.18.0,>=2.14.2->docker->acryl-datahub[postgres]) (4.0.0)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/python3/lib/python3.8/site-packages (from requests!=2.18.0,>=2.14.2->docker->acryl-datahub[postgres]) (2.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/python3/lib/python3.8/site-packages (from requests!=2.18.0,>=2.14.2->docker->acryl-datahub[postgres]) (1.26.2)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/python3/lib/python3.8/site-packages (from requests!=2.18.0,>=2.14.2->docker->acryl-datahub[postgres]) (2020.12.5)
Collecting packaging
  Downloading packaging-21.0-py3-none-any.whl (40 kB)
     |████████████████████████████████| 40 kB 124 kB/s 
Collecting pyparsing>=2.0.2
  Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
     |████████████████████████████████| 67 kB 116 kB/s 
Requirement already satisfied: backports.zoneinfo in /usr/local/python3/lib/python3.8/site-packages (from tzlocal->avro-gen3==0.6.0->acryl-datahub[postgres]) (0.2.1)
Installing collected packages: pyparsing, sqlalchemy, packaging, psycopg2-binary, GeoAlchemy2
Successfully installed GeoAlchemy2-0.9.4 packaging-21.0 psycopg2-binary-2.9.1 pyparsing-2.4.7 sqlalchemy-1.3.24
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
mysql
python3 -m pip install 'acryl-datahub[mysql]'
Requirement already satisfied: acryl-datahub[mysql] in /usr/local/python3/lib/python3.8/site-packages (0.8.14.1)
Requirement already satisfied: pydantic>=1.5.1 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (1.8.2)
Requirement already satisfied: typing-inspect in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (0.7.1)
Requirement already satisfied: stackprinter in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (0.2.5)
Requirement already satisfied: tabulate in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (0.8.9)
Requirement already satisfied: click>=6.0.0 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (8.0.1)
Requirement already satisfied: entrypoints in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (0.3)
Requirement already satisfied: click-default-group in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (1.2.2)
Requirement already satisfied: mypy-extensions>=0.4.3 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (0.4.3)
Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (2.8.2)
Requirement already satisfied: docker in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (4.4.1)
Requirement already satisfied: expandvars>=0.6.5 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (0.7.0)
Requirement already satisfied: toml>=0.10.0 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (0.10.2)
Requirement already satisfied: PyYAML in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (5.3.1)
Requirement already satisfied: avro-python3>=1.8.2 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (1.10.1)
Requirement already satisfied: avro-gen3==0.6.0 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (0.6.0)
Requirement already satisfied: sqlalchemy==1.3.24 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub[mysql]) (1.3.24)
Collecting pymysql>=1.0.2
  Downloading PyMySQL-1.0.2-py3-none-any.whl (43 kB)
     |████████████████████████████████| 43 kB 18 kB/s 
Requirement already satisfied: tzlocal in /usr/local/python3/lib/python3.8/site-packages (from avro-gen3==0.6.0->acryl-datahub[mysql]) (3.0)
Requirement already satisfied: pytz in /usr/local/python3/lib/python3.8/site-packages (from avro-gen3==0.6.0->acryl-datahub[mysql]) (2021.1)
Requirement already satisfied: six in /usr/local/python3/lib/python3.8/site-packages (from avro-gen3==0.6.0->acryl-datahub[mysql]) (1.15.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/python3/lib/python3.8/site-packages (from pydantic>=1.5.1->acryl-datahub[mysql]) (3.10.0.2)
Requirement already satisfied: requests!=2.18.0,>=2.14.2 in /usr/local/python3/lib/python3.8/site-packages (from docker->acryl-datahub[mysql]) (2.25.1)
Requirement already satisfied: websocket-client>=0.32.0 in /usr/local/python3/lib/python3.8/site-packages (from docker->acryl-datahub[mysql]) (0.57.0)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/python3/lib/python3.8/site-packages (from requests!=2.18.0,>=2.14.2->docker->acryl-datahub[mysql]) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/python3/lib/python3.8/site-packages (from requests!=2.18.0,>=2.14.2->docker->acryl-datahub[mysql]) (2020.12.5)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/python3/lib/python3.8/site-packages (from requests!=2.18.0,>=2.14.2->docker->acryl-datahub[mysql]) (1.26.2)
Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/python3/lib/python3.8/site-packages (from requests!=2.18.0,>=2.14.2->docker->acryl-datahub[mysql]) (4.0.0)
Requirement already satisfied: backports.zoneinfo in /usr/local/python3/lib/python3.8/site-packages (from tzlocal->avro-gen3==0.6.0->acryl-datahub[mysql]) (0.2.1)
Installing collected packages: pymysql
Successfully installed pymysql-1.0.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
这个时候mysql和postgresql已经是非disabled的状态了
 python3 -m datahub check plugins
Sources:
athena
azure-ad
bigquery       (disabled)
bigquery-usage (disabled)
datahub-business-glossary
dbt
druid          (disabled)
feast
file
glue           (disabled)
hive           (disabled)
kafka          (disabled)
kafka-connect
ldap           (disabled)
looker         (disabled)
lookml         (disabled)
mongodb        (disabled)
mssql          (disabled)
mysql
okta           (disabled)
oracle         (disabled)
postgres
redash         (disabled)
redshift       (disabled)
sagemaker      (disabled)
snowflake      (disabled)
snowflake-usage(disabled)
sqlalchemy
superset

Sinks:
console
datahub-kafka  (disabled)
datahub-rest
file

Transformers:
add_dataset_ownership
add_dataset_tags
mark_dataset_status
pattern_add_dataset_ownership
set_dataset_browse_path
simple_add_dataset_ownership
simple_add_dataset_tags
simple_remove_dataset_ownership

For details on why a plugin is disabled, rerun with '--verbose'
If a plugin is disabled, try running: pip install 'acryl-datahub[]'

运行第3个的时候有报错，但是再跑的时候好像又没有报错了

python3 -m pip install --upgrade acryl-datahub 
Requirement already satisfied: acryl-datahub in /usr/local/python3/lib/python3.8/site-packages (0.8.14.1)
Requirement already satisfied: docker in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (4.4.1)
Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (2.8.2)
Requirement already satisfied: mypy-extensions>=0.4.3 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (0.4.3)
Requirement already satisfied: toml>=0.10.0 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (0.10.2)
Requirement already satisfied: PyYAML in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (5.3.1)
Requirement already satisfied: avro-python3>=1.8.2 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (1.10.1)
Requirement already satisfied: expandvars>=0.6.5 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (0.7.0)
Requirement already satisfied: click-default-group in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (1.2.2)
Requirement already satisfied: pydantic>=1.5.1 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (1.8.2)
Requirement already satisfied: click>=6.0.0 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (8.0.1)
Requirement already satisfied: typing-inspect in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (0.7.1)
Requirement already satisfied: entrypoints in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (0.3)
Requirement already satisfied: tabulate in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (0.8.9)
Requirement already satisfied: stackprinter in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (0.2.5)
Requirement already satisfied: avro-gen3==0.6.0 in /usr/local/python3/lib/python3.8/site-packages (from acryl-datahub) (0.6.0)
Requirement already satisfied: tzlocal in /usr/local/python3/lib/python3.8/site-packages (from avro-gen3==0.6.0->acryl-datahub) (3.0)
Requirement already satisfied: six in /usr/local/python3/lib/python3.8/site-packages (from avro-gen3==0.6.0->acryl-datahub) (1.15.0)
Requirement already satisfied: pytz in /usr/local/python3/lib/python3.8/site-packages (from avro-gen3==0.6.0->acryl-datahub) (2021.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/python3/lib/python3.8/site-packages (from pydantic>=1.5.1->acryl-datahub) (3.10.0.2)
Requirement already satisfied: requests!=2.18.0,>=2.14.2 in /usr/local/python3/lib/python3.8/site-packages (from docker->acryl-datahub) (2.25.1)
Requirement already satisfied: websocket-client>=0.32.0 in /usr/local/python3/lib/python3.8/site-packages (from docker->acryl-datahub) (0.57.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/python3/lib/python3.8/site-packages (from requests!=2.18.0,>=2.14.2->docker->acryl-datahub) (1.26.2)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/python3/lib/python3.8/site-packages (from requests!=2.18.0,>=2.14.2->docker->acryl-datahub) (2.10)
Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/python3/lib/python3.8/site-packages (from requests!=2.18.0,>=2.14.2->docker->acryl-datahub) (4.0.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/python3/lib/python3.8/site-packages (from requests!=2.18.0,>=2.14.2->docker->acryl-datahub) (2020.12.5)
Requirement already satisfied: backports.zoneinfo in /usr/local/python3/lib/python3.8/site-packages (from tzlocal->avro-gen3==0.6.0->acryl-datahub) (0.2.1)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

是这个样子的

 python3 -m datahub version
DataHub CLI version: 0.8.14.1
Python version: 3.8.0 (default, Jan  4 2021, 16:47:18) 
[GCC 6.3.1 20170216 (Red Hat 6.3.1-3)]

安装docker

python3 -m datahub docker quickstart

 python3 -m datahub docker quickstart 
Fetching docker-compose file from GitHub
Datahub Neo4j volume found, starting with neo4j as graph service.
If you want to run using elastic, run `datahub docker nuke` and re-ingest your data.

Pulling elasticsearch          ... downloading (21.6%)
Pulling elasticsearch-setup    ... waiting
Pulling mysql                  ... waiting
Pulling datahub-gms            ... downloading (19.8%)
Pulling datahub-frontend-react ... downloading (24.5%)
Pulling mysql-setup            ... done
Pulling neo4j                  ... downloading (45.3%)
Pulling zookeeper              ... done
Pulling broker                 ... done
Pulling schema-registry        ... done
Pulling kafka-setup            ... extracting (100.0%)

这几天没有看就变样了啊，连安装方法都不一样了

查看所有镜像
 docker images

1、启动所有容器

docker start $(docker ps -a | awk '{ print $1}' | tail -n +2)

2、关闭所有容器

docker stop $(docker ps -a | awk '{ print $1}' | tail -n +2)

3、删除所有容器

docker rm $(docker ps -a | awk '{ print $1}' | tail -n +2)

4、删除所有镜像（慎用）

docker rmi $(docker images | awk '{print $3}' |tail -n +2)

systemctl status docker 

docker container ls

报错1

 python3 -m datahub docker quickstart 
Fetching docker-compose file from GitHub
No Datahub Neo4j volume found, starting with elasticsearch as graph service.
To use neo4j as a graph backend, run 
`datahub docker quickstart --quickstart-compose-file ./docker/quickstart/docker-compose.quickstart.yml`
from the root of the datahub repo

Pulling elasticsearch          ... done
Pulling elasticsearch-setup    ... done
Pulling mysql                  ... waiting
Pulling datahub-gms            ... waiting
Pulling datahub-frontend-react ... done
Pulling mysql-setup            ... done
Pulling zookeeper              ... done
Pulling broker                 ... done
Pulling schema-registry        ... done
Pulling kafka-setup            ... done

ERROR: for mysql  error pulling image configuration: unknown blob

ERROR: for datahub-gms  error pulling image configuration: unknown blob
ERROR: error pulling image configuration: unknown blob
[2021-09-22 16:20:45,776] ERROR    {datahub.entrypoints:99} - File "/usr/local/python3/lib/python3.8/site-packages/datahub/entrypoints.py", line 91, in main
    88   def main(**kwargs):
    89       # This wrapper prevents click from suppressing errors.
    90       try:
--> 91           sys.exit(datahub(standalone_mode=False, **kwargs))
    92       except click.exceptions.Abort:
    ..................................................
     kwargs = {'prog_name': 'python3 -m datahub'}
     datahub = 
     click.exceptions.Abort = 
    ..................................................

File "/usr/local/python3/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    827  def __call__(self, *args, **kwargs):
 (...)
--> 829      return self.main(*args, **kwargs)
    ..................................................
     self = 
     args = ()
     kwargs = {'standalone_mode': False,
               'prog_name': 'python3 -m datahub'}
    ..................................................

File "/usr/local/python3/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
File "/usr/local/python3/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/python3/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/python3/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/python3/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
File "/usr/local/python3/lib/python3.8/site-packages/datahub/cli/docker.py", line 196, in quickstart
    145  def quickstart(
    146      version: str,
    147      build_locally: bool,
    148      quickstart_compose_file: List[pathlib.Path],
    149      dump_logs_on_failure: bool,
    150      graph_service_impl: Optional[str],
    151  ) -> None:
 (...)
    192          "datahub",
    193      ]
    194  
    195      # Pull and possibly build the latest containers.
--> 196      subprocess.run(
    197          [
    ..................................................
     quickstart = 
     version = 'head'
     build_locally = False
     quickstart_compose_file = [PosixPath('/tmp/tmpttnj9c2j.yml'), ]
     List = typing.List
     pathlib.Path = 
     dump_logs_on_failure = False
     graph_service_impl = None
     Optional = typing.Optional
     subprocess.run = 
    ..................................................

File "/usr/local/python3/lib/python3.8/subprocess.py", line 512, in run
    448  def run(*popenargs,
    449          input=None, capture_output=False, timeout=None, check=False, **kwargs):
 (...)
    508              # We don't call process.wait() as .__exit__ does that for us.
    509              raise
    510          retcode = process.poll()
    511          if check and retcode:
--> 512              raise CalledProcessError(retcode, process.args,
    513                                       output=stdout, stderr=stderr)
    ..................................................
     popenargs = (['docker-compose', '-f', '/tmp/tmpttnj9c2j.yml', '-p', 'datahub', 'pull', ], )
     input = None
     capture_output = False
     timeout = None
     check = True
     kwargs = {}
     retcode = 1
     process.poll =  subprocess.py:1048>
     CalledProcessError = 
     process.args = ['docker-compose', '-f', '/tmp/tmpttnj9c2j.yml', '-p', 'datahub', 'pull', ]
     stdout = None
     stderr = None
    ..................................................

---- (full traceback above) ----
File "/usr/local/python3/lib/python3.8/site-packages/datahub/entrypoints.py", line 91, in main
    sys.exit(datahub(standalone_mode=False, **kwargs))
File "/usr/local/python3/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
File "/usr/local/python3/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
File "/usr/local/python3/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/python3/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/python3/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/python3/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
File "/usr/local/python3/lib/python3.8/site-packages/datahub/cli/docker.py", line 196, in quickstart
    subprocess.run(
File "/usr/local/python3/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,

CalledProcessError: Command '['docker-compose', '-f', '/tmp/tmpttnj9c2j.yml', '-p', 'datahub', 'pull']' returned non-zero exit status 1.

跟以下好像都没有关系，升吧，无所谓

python3 -m pip install pyopenssl ndg-httpsclient pyasn1

python3 -m pip install -U urllib3

python3 -m pip install -U channels

python3 -m pip --cert /Users/user/cacert.pem install -U channels

python3 -m pip install requests[security]

核心问题是这个！！！！docker启用了国内源！！！或者说只有国内源，因此建议暂时关闭docker的国内源！

cp /etc/docker/daemon.json  /etc/docker/daemon.jsonbak
rm /etc/docker/daemon.json 
systemctl restart docker

一切正常

python3 -m datahub docker quickstart 
Fetching docker-compose file from GitHub
No Datahub Neo4j volume found, starting with elasticsearch as graph service.
To use neo4j as a graph backend, run 
`datahub docker quickstart --quickstart-compose-file ./docker/quickstart/docker-compose.quickstart.yml`
from the root of the datahub repo

Pulling elasticsearch          ... done
Pulling elasticsearch-setup    ... done
Pulling mysql                  ... done
Pulling datahub-gms            ... done
Pulling datahub-frontend-react ... done
Pulling mysql-setup            ... done
Pulling zookeeper              ... done
Pulling broker                 ... done
Pulling schema-registry        ... done
Pulling kafka-setup            ... done

Creating network "datahub_network" with the default driver
Creating volume "datahub_esdata" with default driver
Creating volume "datahub_mysqldata" with default driver
Creating volume "datahub_zkdata" with default driver
Creating mysql         ... done
Creating zookeeper     ... done
Creating elasticsearch ... done
Creating datahub-gms         ... done
Creating elasticsearch-setup ... done
Creating broker              ... done
Creating mysql-setup         ... done
Creating datahub-frontend-react ... done
Creating schema-registry        ... done
Creating kafka-setup            ... done
....
elasticsearch is up-to-date
mysql is up-to-date
zookeeper is up-to-date
elasticsearch-setup is up-to-date
mysql-setup is up-to-date
datahub-gms is up-to-date
broker is up-to-date
datahub-frontend-react is up-to-date
schema-registry is up-to-date
kafka-setup is up-to-date
.............
zookeeper is up-to-date
Starting mysql ... 
elasticsearch is up-to-date
broker is up-to-date
elasticsearch-setup is up-to-date
schema-registry is up-to-date
Starting mysql ... done
mysql-setup is up-to-date
datahub-gms is up-to-date
datahub-frontend-react is up-to-date
.........
zookeeper is up-to-date
mysql is up-to-date
elasticsearch is up-to-date
broker is up-to-date
datahub-gms is up-to-date
Starting mysql-setup ... 
Starting elasticsearch-setup ... 
schema-registry is up-to-date
Starting mysql-setup         ... done
Starting elasticsearch-setup ... done
...........
elasticsearch is up-to-date
mysql is up-to-date
Starting elasticsearch-setup ... 
zookeeper is up-to-date
Starting mysql-setup         ... 
datahub-gms is up-to-date
broker is up-to-date
Starting elasticsearch-setup ... done
Starting mysql-setup         ... done
Starting kafka-setup         ... done
.............
zookeeper is up-to-date
mysql is up-to-date
elasticsearch is up-to-date
broker is up-to-date
Starting mysql-setup ... 
datahub-gms is up-to-date
Starting elasticsearch-setup ... 
schema-registry is up-to-date
Starting mysql-setup         ... done
Starting elasticsearch-setup ... done
.......
✔ DataHub is now running
Ingest some demo data using `datahub docker ingest-sample-data`,
or head to http://localhost:9002 (username: datahub, password: datahub) to play around with the frontend.
Need support? Get in touch on Slack: https://slack.datahubproject.io/

截图留念

微信截图_20210922163316.png

你可能感兴趣的:(DataHub 0.8.14.1安装方法)

  • Ubuntu20安装最新nodejs 呵呵你真行 系统集成node.jsnpmjavascript后端ubuntu
    由于Ubuntu20通过apt安装nodejs默认只能到10.xxx版本。最新版本的话需要通过二进制或者源码安装，源码安装需要进行编译耗时较长，本文主要介绍二进制文件安装方法。具体步骤如下：（1）在该目录下找到最新的node版本：https://nodejs.org/dist（2）选择该改机器支持的版本（截止本文章发表前，nodejs最新版本是14.16.1）（3）通过以下命令下载wgethttp
  • 【DataHub】 现代数据栈的元数据平台的Serving Architecture【服务体系架构】讲解 九层之台起于累土 【数据治理】【BigData】架构kafkajava数据库大数据
    ServingArchitecture架构图DataHub服务层的架构图如下：主服务是gms，它提供一套RESTAPI和一套GraphQLAPI，用于对元数据执行CRUD操作元数据服务还提供了搜索和图查询api，以支持二级索引查询、全文搜索以及数据血缘的关系查询服务层组件基本概念:MCE:MetadataChangeEvent元数据更新事件MAE:MetadataCommitLogStream元数
  • 特斯拉神器TeslaMate一键安装，终于来了 oakley0 cartesla云服务器腾讯云
    之前分享了teslamate的功能和简单安装方法，很多喜欢尝鲜的车友尝试了，但安装过程对不熟悉linux服务器的非码农来说还是有点小艰辛。趁这回双十一腾讯云重磅优惠，我也重新屯了服务器重装了一遍，现在把简化后安装过程、一键安装方法包括加密登录的方式分享一下。目录1.购买服务器2.登录服务器3.安装TeslaMate3.1切换管理员用户3.2一键安装TeslaMate-【简单模式】3.3一键安装Te
  • 特斯拉神器TeslaMate一键安装，来了 oakley04 腾讯云阿里云云计算
    之前分享了teslamate的功能和简单安装方法，很多喜欢尝鲜的车友尝试了，但安装过程对不熟悉linux服务器的非码农来说还是有点小艰辛。趁这回双十一腾讯云重磅优惠，我也重新屯了服务器重装了一遍，现在把简化后安装过程或一键安装方法分享一下。1.购买服务器以下三款服务器都可以，其中最推荐中间的2核4G8M带宽的三年198，还没入手请点击下面的入口链接：腾讯云运营活动-腾讯云https://curl.
  • Docker Compose 的安装方 落叶@Henry docker容器运维
    DockerCompose的安装方法有多种，以下是其中三种常见的方法：方法一：使用EPEL源直接安装安装EPEL源：yum-yinstallepel-release安装DockerCompose：yum-yinstalldocker-compose方法二：基于二进制文件安装下载DockerCompose的二进制文件：curl-SL“https://github.com/docker/compose
  • 安全测试工具安装指南：在统信UOS上部署Burp Suite 鹏大圣运维 linux统信UOS安全测试工具gitlinux操作系统国产化桌面
    原文链接：安全测试工具安装指南：在统信UOS上部署BurpSuite大家好！在网络安全领域，BurpSuite是一款不可或缺的工具，它提供了从初级映射和分析应用程序攻击面到查找和利用安全漏洞的一系列功能。今天，我将为大家介绍如何在统信UOS上安装BurpSuiteCommunityEdition（社区版），版本为2023.12.1.5。值得一提的是，社区版和专业版的安装方法基本一致。准备工作在开始
  • FL Studio21.2.3最新版一键安装版专业版水果FLStudio 2024最新下载 yanling2023 macosFLStudio21FLStudio
    FLStudio21.2.3.4004中文破解版系统要求：版本：v21.2.3内部版本[4004]开发商：Image-Line格式：独立、VST位深度：64位界面语言：英语、德语、西班牙语、法语、中文。系统环境MicrosoftWindows1064位（建议在Win10电脑上使用）安装方法1、下载FLStudio编曲软件，如下图所示：FLStudio21链接:百度网盘-链接不存在提取码:07d9F
  • jmeter3.x以上版本安装插件的方法 测试帮日记
    点击链接加入QQ群522720170（免费公开课、视频应有尽有）：https://jq.qq.com/?_wv=1027&k=5C08ATe好多朋友问到jmeter怎么装插件，其实小编是一脸懵逼的。。。。官网已经有了详细的说明，但100个人里会有超过90个问，好吧，那这次就说一下这个问题吧在jmeter3.0之前的版本中，插件的安装方法需要自行去找相应的jar包然后放到对应的目录中。但现在基本不在
  • archlinux安装记录 HayesPan linuxarchlinuxoslinuxarchlinuxos
    摘自http://www.guokr.com/blog/442089/我原来使用Ubuntu，后来受不了它的臃肿才开始用上Archlinux。所以，我是在Ubuntu上覆盖安装Archlinux的。大家实际的安装情况可能会和我的略有不同，这篇Guide就当作是给大家一个参考吧。安装方法:通过硬盘用grub引导ISO(archlinux-2013.01.04-dual.iso,i686)来安装，安装
  • Python cvxpy 安装报错问题 seeseaXi python开发语言线性代数
    学习数学建模的过程中，在线性规划以及非线性规划的章节中，经常会出现要使用cvxpy.solvers模块求解的模型程序，而python当中是没有自带cvxpy这个库的，这意味着我们需要自行安装库。根据网络资料的查询，我得知了：安装cvxpy需要先安装numpy，mkl，scipy，cvxopt，scs，ecos，osqp这几个包至于安装方法，则是通过cmd命令窗口用pip以此安装即可pipinsta
  • Python如何运用matplotlib库绘制3D图形 hakesashou python基础知识pythonmatplotlib3d
    3D图形在数据分析、数据建模、图形和图像处理等领域中都有着广泛的应用，下面将给大家介绍一下如何在Python中使用matplotlib进行3D图形的绘制，包括3D散点、3D表面、3D轮廓、3D直线（曲线）以及3D文字等的绘制。准备工作：python中绘制3D图形，依旧使用常用的绘图模块matplotlib，但需要安装mpl_toolkits工具包，安装方法如下：windows命令行进入到pytho
  • 清华 armbian bookworm 源 linux实践操作记录 阿里云服务器云计算
    Debian12(bookworm)安装方法及其国内软件源开源地理空间基金会中文分会开放地理空间实验室(osgeo.cn)vi/etc/apt/sources.listdebhttps://mirrors.tuna.tsinghua.edu.cn/debian/bookwormmainnon-freenon-free-firmwarecontribdeb-srchttps://mirrors.tu
  • yum简易安装的k8s 爱写代码的菜鸟
    在阿里云的ECS下（centos系统）安装k8s相信部署过或者了解过k8s部署的同学都知道，k8s很好但是安装起来比较复杂，所以给大家整理了如下快捷的安装方法1.yuminstall-yetcdkubernetes2.vim/etc/sysconfig/docker#内容改为如下OPTIONS='--selinux-enabled=false--insecure-registrygcr.io'3.
  • Chrome浏览器插件axure-chrome-extension（免费） 兮风萧萧 chromeaxure前端
    点我下载插件AxureRPExtensionforChrome是原型设计工具AxureRP的Chrome浏览器插件。因为在线安装需要访问GoogleChrome在线商店，访问不了。所以提供一个离线版本进行安装。插件内容如下：安装方法：1、打开Chrome浏览器，找到“工具->扩展程序”，打开开发者模式；2、将下载的axure-chrome-extension.zip文件解压，点击加载已解压的扩展程
  • axure-chrome-extension 插件安装 嘻嘻的妙妙屋 微信小程序前端chromeaxure前端
    场景在我们收到axure-chrome-extension画的界面，通过浏览器无法打开时，在网上又无法找到安装包；安装方法我们打开页面原型的路径下的resources的chrome目录里面有axure-chrome-extension.crx复制一份更名为rar解压之后打开谷歌浏览器打开扩展程序添加扩展程序后面就能正常访问页面原型了下载地址附带插件下载地址，按照上面的方式即可安装：https://
  • 教你如何正确的安装官方的驱动程序 亿说电脑
    大家都知道我们组装一台电脑的首先是装机，然后装系统，最后装驱动，这里我们就谈谈如何安装正确的驱动程序。为什么说是安装“正确”的驱动程序呢，驱动程序难道还有错误的吗？很多人说装驱动还要亲自装吗，现在不都是傻瓜式的一键安装系统，安装完了我就可以用了啊！如果有人这么问，那只能说这个人根本不会装驱动，甚至连驱动程序是什么都不知道吧，就算搞好了驱动也确实能用了，但用的都是非官方的安装方法，在使用当中会出现蓝
  • 基于Ubuntu20.04 Linux系统安装 Microsoft Edge浏览器 详细步骤 cugautozp linuxlinuxubuntuedge
    目录1.MicrosoftEdge2.Linux版本的Edge3.官方第二种安装方法——命令行安装4.总结注意：最近在学习强度学习，作为刚起步的小白啥也不懂就先安装了最新的Ubuntu20.04，听很多人说这个版本相对之前的版本可以很方便的链接网络，个人感觉还不错。  个人之前一直使用Windows系统，初次接触Linux有不足之处请读者多多包含。在Windows系统的时候使用的浏览器，本人还是比
  • livereload 前端开发可视化工具（前端修改，自动刷新） 微little
    安装步骤一谷歌商店搜索livereload，安装谷歌商店二全局安装livereload或者在开发工具安装插件全局安装方法cnpminstall-glivereload在开发工具安装（本例使用的是vscode）示例图三使用在当前文件目录下执行,就可以实现自动刷新了vscode
  • OpenJDK与JDK的区别及Ubuntu下的安装方法 向晚流年 Linux~每天一点点儿
    OpenJDK与JDK的区别：OpenJDK是JDK的开放原始码版本，以GPL协议的形式放出。两者的授权协议的不同，且在采用GPL协议的OpenJDK中，SUNJDK的一部分源代码因为产权的问题无法开放OpenJDK使用，随意相比于JDK，OpenJDK源代码不完整，只包含最精简的JDK。Ubuntu下OpenJDK与JDK的安装：1、OpenJDK的安装参考官方网站的介绍Howtodownloa
  • 【linux系统体验】-ubuntu简易折腾 桃子酱紫君 笔记本儿linuxubuntu运维
    ubuntu一、终端美化二、桌面美化2.1插件安装2.2主题和图标2.3美化配置三、常用命令以后看不看不重要，咱就是想记点儿东西。一、终端美化安装ohmyposh，参考链接：Linux终端美化1、安装字体ohmyposh美化工具可以使用合适的字体，如AgaveNerdFont。安装后配置到终端，如果不使用该字体，可能有些图标无法显示。安装方法：解压后将.ttf文件放入~/.fonts下即可。2、可
  • 苹果M1芯片Mac安装Adobe系列软件安装方法 m1芯片安装Adobe系列软件问题解决方案 小柯基分享
    小编对摄影一直有浓厚的兴趣，这么多年以来就一直使用苹果mac系统做修图！也算是个mac忠实粉丝了。就在11月份,苹果发布了基于AppleSiliconM1SoC芯片打造新Mac系列产品，全新的M1处理器的诞生带来了很多革命性的变化，因为这标志着苹果正式开启了从Intel的x86CPU过渡到该公司自己基于Arm架构设计！这次因为非常看重M1芯片的性能提升，所以一上市就毫不犹豫的购买了新款搭载M1芯片
  • Linux环境下conda安装opencv 迷路在代码中 环境配置图像处理opencvanacondaopencvlinuxanacondapython
    输入指令进行安装：pipinstallopencv-pythonorpipinstall-ihttp://pypi.douban.com/simpleopencv-python--trusted-hostpypi.douban.com然后，通过condalist查看是否安装成功Reference:Anaconda环境下OpenCV的三种安装方法Linux使用Anaconda安装opencv在ana
  • Fit-hic 2 通过hic数据检测显著性互作 寒山梦绮
    安装官网中提到三种安装方法，可通过conda,pip，gitclone安装该软件.我们这里利用conda,非常简单快速.1.1conda创建环境condacreate-nfithicpython=3.81.2conda安装fithiccondainstallfithic1.3测试fithic-V1.4测试数据svnexporthttps://github.com/ay-lab/fithic/tru
  • Hyper-V中安装CentOS7设置静态ip并且可以连接外网 水木小窝 LinuxHyper-VLinuxCentOS
    在Hyper-V中安装好CentOS7，安装的时候配置网络选择Hyper-V默认的交换机（具体安装方法自行查找，这里不再进行说明）对Hyper-V添加一个内部交换机，名称自定义。添加完之后到电脑的网络连接里面对这个交换机设置静态ip。将虚拟机关闭，选择这台虚拟机进行设置，选择添加硬件——>网络适配器——>添加，然后选择新添加的网络适配器选择刚才新建的交换机启动虚拟机，查看ip会看见两个网卡eth0
  • Mamba 环境安装踩坑问题汇总及解决方法 yyywxk #Python模块有关问题Mambapythonpytorch
    项目场景：最近Mamba有关的论文引起了众多人的关注，虽然Mamba论文自身被ICLR2024拒稿，但是其衍生的模型层出不穷，诸如Vim和Umamba等。笔者在配置相关环境（版本安装要求：PyTorch1.12+；CUDA11.6+）时，发现按照他们给的安装方法12安装时会遇到非常多的bug，主要集中在causal-conv1d和mamba-ssm上，原因都是版本兼容问题，特此记录。问题描述直接p
  • Docker是什么（Docker-CE） 勇敢的大蜗牛 运维linuxdocker
    文章目录简介为什么选择DockerDocker优势和虚拟机的区别Docker的使用场景名称空间Docker的核心概念（重点）控制组(Controlgroups)CentOS中的安装方法二、Docker相关命令1、安装依赖包2、设置阿里云镜像源3、安装Docker-CE简介-Docker是一个开源的应用容器引擎，让开发者可以打包他们的应用以及依赖包到一个可移植的容器中,然后发布到任何流行的Linux
  • 如何用 python 获取实时的股票数据？ 程序员小猴紫 Python开发编程python开发语言Python入门编程学习
    这个我会！先上个展示图这篇回答中，我将向你展示两种不同的代码版本（加强版和一般版）代码运行环境说明（非常重要）Python版本要求Python3需要安装的库efinance库的安装方法是：打开cmd（命令提示符或者其他终端工具），输入以下代码pipinstallefinance输入完毕，按Enter键执行代码，等待successfully出现即可代码展示加强版代码这一部分基于我开发的python库
  • Electron基本介绍 灬沙海灬 electron
    Electron基本介绍Electron官方网站：https://www.electronjs.org/zh/Electron安装方法：npminstallelectron-g全局安装+Electron简介：Electron提供了丰富的本地（操作系统）API，使你能够使用纯JavaScript来创建桌面应用程序。Electron通过集成浏览器内核，使用Web技术来实现不同平台下的渲染，并结合了Ch
  • Electron基本介绍 记忆的小河 javascriptnode.js
    Electron基本介绍Electron官方网站：https://www.electronjs.org/zh/Electron安装方法：npminstallelectron-g全局安装+Electron简介：Electron提供了丰富的本地（操作系统）API，使你能够使用纯JavaScript来创建桌面应用程序。Electron通过集成浏览器内核，使用Web技术来实现不同平台下的渲染，并结合了Ch
  • BaiduNetdiskPlugin Mac版百度网盘超级会员插件 产品U理
    这次Mac毒为大家带来一款百度网盘客户端的超级会员插件，可以开启无限免费试用功能，速度杠杠的，需要的小伙伴可以试试，安装方法也很简单。由于本教程是基于百度网盘官方客户端而进行破解的，所以小伙伴们请务必先下载官方版的客户端哦，官方下载地址「https://pan.baidu.com/download」。安装教程1.下载并安装完成百度网盘客户端后，打开Mac的「终端」应用。不知道在哪里的可以按快捷键「
  • Enum用法 不懂事的小屁孩 enum
    以前的时候知道enum，但是真心不怎么用，在实际开发中，经常会用到以下代码: protected final static String XJ = "XJ"; protected final static String YHK = "YHK"; protected final static String PQ = "PQ";
  • 【Spark九十七】RDD API之aggregateByKey bit1129 spark
    1. aggregateByKey的运行机制   /** * Aggregate the values of each key, using given combine functions and a neutral "zero value". * This function can return a different result type
  • hive创建表是报错： Specified key was too long; max key length is 767 bytes daizj hive
    今天在hive客户端创建表时报错，具体操作如下   hive> create table test2(id string); FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataSto
  • Map 与 JavaBean之间的转换 周凡杨 java自省转换反射
    最近项目里需要一个工具类，它的功能是传入一个Map后可以返回一个JavaBean对象。很喜欢写这样的Java服务，首先我想到的是要通过Java 的反射去实现匿名类的方法调用，这样才可以把Map里的值set 到JavaBean里。其实这里用Java的自省会更方便，下面两个方法就是一个通过反射，一个通过自省来实现本功能。 1：JavaBean类 1 &nb
  • java连接ftp下载 g21121 java
    有的时候需要用到java连接ftp服务器下载，上传一些操作，下面写了一个小例子。 /** ftp服务器地址 */ private String ftpHost; /** ftp服务器用户名 */ private String ftpName; /** ftp服务器密码 */ private String ftpPass; /** ftp根目录 */ private String f
  • web报表工具FineReport使用中遇到的常见报错及解决办法（二） 老A不折腾 finereportweb报表java报表总结
      抛砖引玉，希望大家能把自己整理的问题及解决方法晾出来，Mark一下，利人利己。   出现问题先搜一下文档上有没有，再看看度娘有没有，再看看论坛有没有。有报错要看日志。下面简单罗列下常见的问题，大多文档上都有提到的。   1、没有返回数据集： 在存储过程中的操作语句之前加上set nocount on 或者在数据集exec调用存储过程的前面加上这句。当S
  • linux 系统cpu 内存等信息查看 墙头上一根草 cpu内存liunx
    1 查看CPU 　　1.1 查看CPU个数 　　# cat /proc/cpuinfo | grep "physical id" | uniq | wc -l 　　2 　　**uniq命令：删除重复行;wc –l命令：统计行数** 　　1.2 查看CPU核数 　　# cat /proc/cpuinfo | grep "cpu cores" | u
  • Spring中的AOP aijuans springAOP
      Spring中的AOP Written by Tony Jiang @ 2012-1-18 （转）何为AOP AOP，面向切面编程。 在不改动代码的前提下，灵活的在现有代码的执行顺序前后，添加进新规机能。 来一个简单的Sample: 目标类： [java]  view plain copy print ? package&nb
  • placeholder(HTML 5) IE 兼容插件 alxw4616 JavaScriptjquery jQuery插件
    placeholder 这个属性被越来越频繁的使用. 但为做HTML 5 特性IE没能实现这东西. 以下的jQuery插件就是用来在IE上实现该属性的. /** * [placeholder(HTML 5) IE 实现.IE9以下通过测试.] * v 1.0 by oTwo 2014年7月31日 11:45:29 */ $.fn.placeholder = function
  • Object类,值域,泛型等总结(适合有基础的人看) 百合不是茶 泛型的继承和通配符变量的值域Object类转换
    java的作用域在编程的时候经常会遇到,而我经常会搞不清楚这个 问题,所以在家的这几天回忆一下过去不知道的每个小知识点   变量的值域;   package 基础; /** * 作用域的范围 * * @author Administrator * */ public class zuoyongyu { public static vo
  • JDK1.5 Condition接口 bijian1013 javathreadConditionjava多线程
    Condition 将 Object 监视器方法（wait、notify和 notifyAll）分解成截然不同的对象，以便通过将这些对象与任意 Lock 实现组合使用，为每个对象提供多个等待 set （wait-set）。其中，Lock 替代了 synchronized 方法和语句的使用，Condition 替代了 Object 监视器方法的使用。 条件（也称为条件队列或条件变量）为线程提供了一
  • 开源中国OSC源创会记录 bijian1013 hadoopsparkMemSQL
    一.Strata+Hadoop World（SHW）大会         是全世界最大的大数据大会之一。SHW大会为各种技术提供了深度交流的机会，还会看到最领先的大数据技术、最广泛的应用场景、最有趣的用例教学以及最全面的大数据行业和趋势探讨。          二.Hadoop   &nbs
  • 【Java范型七】范型消除 bit1129 java
    范型是Java1.5引入的语言特性，它是编译时的一个语法现象，也就是说，对于一个类，不管是范型类还是非范型类，编译得到的字节码是一样的，差别仅在于通过范型这种语法来进行编译时的类型检查，在运行时是没有范型或者类型参数这个说法的。 范型跟反射刚好相反，反射是一种运行时行为，所以编译时不能访问的变量或者方法(比如private)，在运行时通过反射是可以访问的，也就是说，可见性也是一种编译时的行为，在
  • 【Spark九十四】spark-sql工具的使用 bit1129 spark
    spark-sql是Spark bin目录下的一个可执行脚本，它的目的是通过这个脚本执行Hive的命令，即原来通过 hive>输入的指令可以通过spark-sql>输入的指令来完成。 spark-sql可以使用内置的Hive metadata-store，也可以使用已经独立安装的Hive的metadata store   关于Hive build into Spark
  • js做的各种倒计时 ronin47 js 倒计时
    第一种：精确到秒的javascript倒计时代码      HTML代码:   <form name="form1">   <div align="center" align="middle"
  • java-37.有n 个长为m+1 的字符串，如果某个字符串的最后m 个字符与某个字符串的前m 个字符匹配，则两个字符串可以联接 bylijinnan java
    public class MaxCatenate { /* * Q.37 有n 个长为m+1 的字符串，如果某个字符串的最后m 个字符与某个字符串的前m 个字符匹配，则两个字符串可以联接， * 问这n 个字符串最多可以连成一个多长的字符串，如果出现循环，则返回错误。 */ public static void main(String[] args){
  • mongoDB安装 开窍的石头 mongodb安装 基本操作
    mongoDB的安装          1:mongoDB下载   https://www.mongodb.org/downloads         2:下载mongoDB下载后解压     
  • [开源项目]引擎的关键意义 comsci 开源项目
         一个系统，最核心的东西就是引擎。。。。。       而要设计和制造出引擎，最关键的是要坚持。。。。。。       现在最先进的引擎技术，也是从莱特兄弟那里出现的，但是中间一直没有断过研发的    
  • 软件度量的一些方法 cuiyadll 方法
    软件度量的一些方法http://cuiyingfeng.blog.51cto.com/43841/6775/在前面我们已介绍了组成软件度量的几个方面。在这里我们将先给出关于这几个方面的一个纲要介绍。在后面我们还会作进一步具体的阐述。当我们不从高层次的概念级来看软件度量及其目标的时候，我们很容易把这些活动看成是不同而且毫不相干的。我们现在希望表明他们是怎样恰如其分地嵌入我们的框架的。也就是我们度量的
  • XSD中的targetNameSpace解释 darrenzhu xmlnamespacexsdtargetnamespace
    参考链接: http://blog.csdn.net/colin1014/article/details/357694 xsd文件中定义了一个targetNameSpace后，其内部定义的元素，属性，类型等都属于该targetNameSpace,其自身或外部xsd文件使用这些元素，属性等都必须从定义的targetNameSpace中找： 例如：以下xsd文件，就出现了该错误，即便是在一
  • 什么是RAID0、RAID1、RAID0+1、RAID5，等磁盘阵列模式? dcj3sjt126com raid
    RAID 1又称为Mirror或Mirroring，它的宗旨是最大限度的保证用户数据的可用性和可修复性。 RAID 1的操作方式是把用户写入硬盘的数据百分之百地自动复制到另外一个硬盘上。由于对存储的数据进行百分之百的备份，在所有RAID级别中，RAID 1提供最高的数据安全保障。同样，由于数据的百分之百备份，备份数据占了总存储空间的一半，因而，Mirror的磁盘空间利用率低，存储成本高。 Mir
  • yii2 restful web服务快速入门 dcj3sjt126com PHPyii2
    快速入门 Yii 提供了一整套用来简化实现 RESTful 风格的 Web Service 服务的 API。 特别是，Yii 支持以下关于 RESTful 风格的 API： 支持 Active Record 类的通用API的快速原型 涉及的响应格式（在默认情况下支持 JSON 和 XML) 支持可选输出字段的定制对象序列化 适当的格式的数据采集和验证错误
  • MongoDB查询(3)——内嵌文档查询（七） eksliang MongoDB查询内嵌文档MongoDB查询内嵌数组
    MongoDB查询内嵌文档 转载请出自出处：http://eksliang.iteye.com/blog/2177301 一、概述        有两种方法可以查询内嵌文档：查询整个文档；针对键值对进行查询。这两种方式是不同的，下面我通过例子进行分别说明。   二、查询整个文档 例如:有如下文档 db.emp.insert({ &qu
  • android4.4从系统图库无法加载图片的问题 gundumw100 android
    典型的使用场景就是要设置一个头像，头像需要从系统图库或者拍照获得，在android4.4之前，我用的代码没问题，但是今天使用android4.4的时候突然发现不灵了。baidu了一圈，终于解决了。 下面是解决方案： private String[] items = new String[] { "图库","拍照" }; /* 头像名称 */
  • 网页特效大全 jQuery等 ini JavaScriptjquerycsshtml5ini
    HTML5和CSS3知识和特效 asp.net ajax jquery实例 分享一个下雪的特效 jQuery倾斜的动画导航菜单 选美大赛示例 你会选谁 jQuery实现HTML5时钟 功能强大的滚动播放插件JQ-Slide 万圣节快乐！！！ 向上弹出菜单jQuery插件 htm5视差动画 jquery将列表倒转顺序 推荐一个jQuery分页插件 jquery animate
  • swift objc_setAssociatedObject block(version1.2 xcode6.4) 啸笑天 version
      import UIKit class LSObjectWrapper: NSObject { let value: ((barButton: UIButton?) -> Void)? init(value: (barButton: UIButton?) -> Void) { self.value = value
  • Aegis 默认的 Xfire 绑定方式，将 XML 映射为 POJO MagicMa_007 javaPOJOxmlAegisxfire
          Aegis 是一个默认的 Xfire 绑定方式，它将 XML 映射为 POJO, 支持代码先行的开发.你开发服 务类与 POJO,它为你生成 XML schema/wsdl XML 和 注解映射概览       默认情况下，你的 POJO 类被是基于他们的名字与命名空间被序列化。如果
  • js get max value in (json) Array qiaolevip 每天进步一点点学习永无止境max纵观千象
    // Max value in Array var arr = [1,2,3,5,3,2];Math.max.apply(null, arr); // 5 // Max value in Jaon Array var arr = [{"x":"8/11/2009","y":0.026572007},{"x"
  • XMLhttpRequest 请求 XML,JSON ,POJO 数据 Luob. POJOjsonAjaxxmlXMLhttpREquest
    在使用XMlhttpRequest对象发送请求和响应之前，必须首先使用javaScript对象创建一个XMLHttpRquest对象。 var xmlhttp； function getXMLHttpRequest(){ if(window.ActiveXObject){ xmlhttp:new ActiveXObject("Microsoft.XMLHTTP
  • jquery wuai jquery
    以下防止文档在完全加载之前运行Jquery代码，否则会出现试图隐藏一个不存在的元素、获得未完全加载的图像的大小 等等 $(document).ready(function(){ jquery代码; }); <script type="text/javascript" src="c:/scripts/jquery-1.4.2.min.js&quo
按字母分类： ABCDEFGHIJKLMNOPQRSTUVWXYZ其他