map和flatMap之间的区别

    scala> rdd5.flatMap(t=>{t.split("\t")}).collect
    res8: Array[String] = Array(75, 2018-09-17 19:15:34.0, BK-HTML5-JY-1819, BK181713017, 张帅杰, 2018-09-17 19:50:37.0, 34, 0, 55, 55, 75, 2018-09-17 19:16:07.0, BK-HTML5-JY-1819, BK181913016, 王润芝, 2018-09-17 19:39:30.0, 23, 0, 62, 62, 75, 2018-09-17 19:16:08.0, BK-HTML5-JY-1819, BK181913062, 唐颖, 2018-09-17 19:39:31.0, 23, 0, 60, 60, 75, 2018-09-17 19:16:47.0, BK-HTML5-JY-1819, BK181913007, 周营营, 2018-09-17 19:43:32.0, 0, 0, 61, 61)
    
scala>  rdd5.map(t=>{t.split("\t")}).collect
res9: Array[Array[String]] = Array(Array(75, 2018-09-17 19:15:34.0, BK-HTML5-JY-1819, BK181713017, 张帅杰, 2018-09-17 19:50:37.0, 34, 0, 55, 55), Array(75, 2018-09-17 19:16:07.0, BK-HTML5-JY-1819, BK181913016, 王润芝, 2018-09-17 19:39:30.0, 23, 0, 62, 62), Array(75, 2018-09-17 19:16:08.0, BK-HTML5-JY-1819, BK181913062, 唐颖, 2018-09-17 19:39:31.0, 23, 0, 60, 60), Array(75, 2018-09-17 19:16:47.0, BK-HTML5-JY-1819, BK181913007, 周营营, 2018-09-17 19:43:32.0, 0, 0, 61, 61))

flatmap是将输入的数据首先进行map操作,然后再进行压平操作,从上面可以看出经过map操作过后的结果是Array(Array()…) , 压平就是将Array中的数组都变成元素Array(…)

你可能感兴趣的:(spark)