6.redis 异常 redis.clients.jedis.exceptions.JedisDataException: MISCONF Redis is configured to
在linux Centos环境下,Java连接redis发出的异常信息如下: redis.clients.jedis.exceptions.JedisDataException: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error. 解决办法: redis 安装目录下 找到 redis-cli.EXE 连接到服务器后执行以下命令: config set stop-writes-on-bgsave-error no 执行结果: ok.
List list = Arrays.asList(items.split("_"));
req_itemids = new ArrayList<>(list);
for (int i = 1; i < req_itemids.size(); i++) {
Float score = Float.parseFloat(req_itemids.get(i).split(":")[1]);
guiyi_sum+=Math.pow(score,2);
if(score<0.5){
req_itemids.remove(i--);
}
}
8.使用比较器对list的对象排序报错java.lang.IllegalArgumentException: Comparison method violates its general contract!
[root@master java]# systemctl status cloudera-scm-server
● cloudera-scm-server.service - Cloudera CM Server Service
Loaded: loaded (/usr/lib/systemd/system/cloudera-scm-server.service; enabled; vendor preset: disabled)
Active: active (running) since 五 2022-03-18 12:24:03 CST; 5min ago
Process: 5510 ExecStartPre=/opt/cloudera/cm/bin/cm-server-pre (code=exited, status=0/SUCCESS)
Main PID: 5512 (java)
CGroup: /system.slice/cloudera-scm-server.service
└─5512 /usr/java/default/bin/java -cp .:/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:lib/* -serv...
3月 18 12:24:03 master systemd[1]: Starting Cloudera CM Server Service...
3月 18 12:24:03 master systemd[1]: Started Cloudera CM Server Service.
3月 18 12:24:03 master cm-server[5512]: JAVA_HOME=/usr/java/default
3月 18 12:24:03 master cm-server[5512]: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
3月 18 12:24:07 master cm-server[5512]: ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system proper...tion logging.
3月 18 12:24:17 master cm-server[5512]: 12:24:17.894 [main] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Table 'scm.CM_VERSION' doesn't exist
Hint: Some lines were ellipsized, use -l to show in full.
日志:
[root@master cloudera]# tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
2022-03-18 12:24:46,099 INFO main:com.cloudera.cmon.kaiser.csd.CsdInfoBasedHealthTestDescriptors: Built CSD-based test descriptor FAILED_DATA_DIRS with scope KUDU-KUDU_TSERVER
2022-03-18 12:24:46,099 WARN main:com.cloudera.cmon.kaiser.csd.CsdInfoBasedHealthTestDescriptors: Duplicate health test KUDU-6.1.0-FAILED_DATA_DIRS from CSD KUDU6_1-6.3.1.
2022-03-18 12:24:46,100 INFO main:com.cloudera.cmon.kaiser.csd.CsdInfoBasedHealthTestDescriptors: Registered all CSD-based health tests for KUDU from CSD KUDU6_1-6.3.1
2022-03-18 12:24:46,100 INFO main:com.cloudera.cmon.kaiser.csd.CsdInfoBasedHealthTestDescriptors: Built CSD-based test descriptor FULL_DATA_DIRS with scope KUDU-KUDU_MASTER
2022-03-18 12:24:46,100 INFO main:com.cloudera.cmon.kaiser.csd.CsdInfoBasedHealthTestDescriptors: Built CSD-based test descriptor FAILED_DATA_DIRS with scope KUDU-KUDU_MASTER
2022-03-18 12:24:46,100 INFO main:com.cloudera.cmon.kaiser.csd.CsdInfoBasedHealthTestDescriptors: Built CSD-based test descriptor FULL_DATA_DIRS with scope KUDU-KUDU_TSERVER
2022-03-18 12:24:46,100 WARN main:com.cloudera.cmon.kaiser.csd.CsdInfoBasedHealthTestDescriptors: Duplicate health test KUDU-6.2.0-FULL_DATA_DIRS from CSD KUDU6_2-6.3.1.
2022-03-18 12:24:46,101 INFO main:com.cloudera.cmon.kaiser.csd.CsdInfoBasedHealthTestDescriptors: Built CSD-based test descriptor FAILED_DATA_DIRS with scope KUDU-KUDU_TSERVER
2022-03-18 12:24:46,101 WARN main:com.cloudera.cmon.kaiser.csd.CsdInfoBasedHealthTestDescriptors: Duplicate health test KUDU-6.2.0-FAILED_DATA_DIRS from CSD KUDU6_2-6.3.1.
2022-03-18 12:24:46,101 INFO main:com.cloudera.cmon.kaiser.csd.CsdInfoBasedHealthTestDescriptors: Registered all CSD-based health tests for KUDU from CSD KUDU6_2-6.3.1
2022-03-18 12:24:46,581 INFO main:com.cloudera.server.cmf.HeartbeatRequester: Eager heartbeat initialized
23.flink作业在standalone提交运行报错
异常信息:
2022-03-20 00:41:23
org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find a file system implementation for scheme 'hdfs'. The scheme is not directly supported by Flink and no Hadoop file system to support this scheme could be loaded. For a full list of supported file systems, please see https://ci.apache.org/projects/flink/flink-docs-stable/ops/filesystems/.
at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:491)
at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:389)
at org.apache.flink.core.fs.Path.getFileSystem(Path.java:292)
at org.apache.flink.runtime.state.filesystem.FsCheckpointStorageAccess.(FsCheckpointStorageAccess.java:64)
at org.apache.flink.runtime.state.filesystem.FsStateBackend.createCheckpointStorage(FsStateBackend.java:501)
at org.apache.flink.streaming.runtime.tasks.StreamTask.(StreamTask.java:302)
at org.apache.flink.streaming.runtime.tasks.StreamTask.(StreamTask.java:277)
at org.apache.flink.streaming.runtime.tasks.StreamTask.(StreamTask.java:257)
at org.apache.flink.streaming.runtime.tasks.StreamTask.(StreamTask.java:250)
at org.apache.flink.streaming.runtime.tasks.StreamTask.(StreamTask.java:240)
at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.(OneInputStreamTask.java:65)
at sun.reflect.GeneratedConstructorAccessor13.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.flink.runtime.taskmanager.Task.loadAndInstantiateInvokable(Task.java:1373)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:700)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:547)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Hadoop is not in the classpath/dependencies.
at org.apache.flink.core.fs.UnsupportedSchemeFactory.create(UnsupportedSchemeFactory.java:58)
at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:487)
... 17 more
运行命令: [root@bigdata101 hadoop]# hadoop jar /usr/local/src/hadoop-3.1.3/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.3.jar wordcount /input /output1 报错显示:
2022-03-22 14:39:47,825 INFO mapreduce.Job: map 0% reduce 0%
2022-03-22 14:39:47,862 INFO mapreduce.Job: Job job_1647930942134_0004 failed with state FAILED due to: Application application_1647930942134_0004 failed 2 times due to AM Container for appattempt_1647930942134_0004_000002 exited with exitCode: 1
Failing this attempt.Diagnostics: [2022-03-22 14:39:47.108]Exception from container-launch.
Container id: container_1647930942134_0004_02_000001
Exit code: 1
[2022-03-22 14:39:47.112]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
错误: 找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster
[2022-03-22 14:39:47.113]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
错误: 找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster
For more detailed output, check the application tracking page: http://bigdata101:8088/cluster/app/application_1647930942134_0004 Then click on links to logs of each attempt.
. Failing the application.
2022-03-22 14:39:47,905 INFO mapreduce.Job: Counters: 0
解决办法: 在主机中运行: hadoop classpath
记下返回的结果 vi $HADOOP_HOME/etc/hadoop/yarn-site.xml 添加一个配置
yarn.application.classpathhadoop classpath返回信息
重启yarn
25.执行Hadoop的mapreduce的jar包出现 name node is in safe mode 问题
解决方法: 1.进入hadoop安装根目录 执行 cd /usr/local/hadoop bin/hadoop dfsadmin -safemode leave
java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1679)
at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:221)
at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:572)
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:412)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1679)
at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:221)
at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:572)
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:412)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
at java.lang.Thread.run(Thread.java:748)
以教员和课程为例介绍一对多关联关系,在这里认为一个教员可以叫多门课程,而一门课程只有1个教员教,这种关系在实际中不太常见,通过教员和课程是多对多的关系。
示例数据:
地址表:
CREATE TABLE ADDRESSES
(
ADDR_ID INT(11) NOT NULL AUTO_INCREMENT,
STREET VAR
In this lesson we used the key "UITextAttributeTextColor" to change the color of the UINavigationBar appearance to white. This prompts a warning "first deprecated in iOS 7.0."
Ins
质数也叫素数,是只能被1和它本身整除的正整数,最小的质数是2,目前发现的最大的质数是p=2^57885161-1【注1】。
判断一个数是质数的最简单的方法如下:
def isPrime1(n):
for i in range(2, n):
if n % i == 0:
return False
return True
但是在上面的方法中有一些冗余的计算,所以
hbase(hadoop)是用java编写的,有些语言(例如python)能够对它提供良好的支持,但也有很多语言使用起来并不是那么方便,比如c#只能通过thrift访问。Rest就能很好的解决这个问题。Hbase的org.apache.hadoop.hbase.rest包提供了rest接口,它内嵌了jetty作为servlet容器。
启动命令:./bin/hbase rest s
下面这段sql本来目的是想更新条件下的数据,可是这段sql却更新了整个表的数据。sql如下:
UPDATE tops_visa.visa_order
SET op_audit_abort_pass_date = now()
FROM
tops_visa.visa_order as t1
INNER JOIN tops_visa.visa_visitor as t2
ON t1.