pyflink的local模式实验记录

python和flink相关的网上例子比较少

先温固下shell中这些环境变量吧。

python  flink shell变量	类型	scala flink shell变量	类型
s_env	pyflink.datastream.stream_execution_environment.StreamExecutionEnvironment	senv	StreamExecutionEnvironment
st_env	pyflink.table.table_environment.StreamTableEnvironment	stenv	StreamTableEnvironmentImpl
b_env	class 'pyflink.dataset.execution_environment.ExecutionEnvironment	benv	ExecutionEnvironment
bt_env	pyflink.table.table_environment.BatchTableEnvironment	btenv	BatchTableEnvironmentImpl

注:

scala获取类型举例:

scala> senv.getClass.getSimpleName
res2: String = StreamExecutionEnvironment

####################################################################################################

启动命令:

pyflink-shell.sh local

 交互模式中输入的代码(Python DataStream API):

import tempfile
import os
import shutil
sink_path = tempfile.gettempdir() + '/streaming.csv'
if os.path.exists(sink_path):
    if os.path.isfile(sink_path):
        os.remove(sink_path)
    else:
        shutil.rmtree(sink_path)

s_env.set_parallelism(1)
t = st_env.from_elements([(1, 'hi', 'hello'), (2, 'hi', 'hello')], ['a', 'b', 'c'])
st_env.connect(FileSystem().path(sink_path)).with_format(OldCsv()
                 .field_delimiter(',')
                 .field("a", DataTypes.BIGINT())
                 .field("b", DataTypes.STRING())
                 .field("c", DataTypes.STRING())).with_schema(Schema()
                 .field("a", DataTypes.BIGINT())
                 .field("b", DataTypes.STRING())
                 .field("c", DataTypes.STRING())).register_table_sink("stream_sink")

t.select("a + 1, b, c").insert_into("stream_sink")

st_env.execute("stream_job")

实验结果:

/tmp/streaming.csv

打开后如下: