Spark2.x Session无法使用UDTF内置函数问题
select
t.valid,
t.remote_addr,
t.remote_user,
t.time_local,
substring(t.time_local,0,10),
substring(t.time_local,11),
substring(t.time_local,6,2),
substring(t.time_local,9,2),
substring(t.time_local,12,2),
t.request,
t.status,
t.body_bytes_sent,
t.http_referer,
t.host,
t.path,
t.query,
t.query_id,
t.http_user_agent
from(
select
a.*,
b.*
from qm1711.ods_weblog_origin a
lateral view parse_url_tuple(regexp_replace(http_referer,'\"',''),'HOST','PATH','QUERY','QUERY:id') b as host,path,query,query_id
)t;"
用SparkSession 执行这句的时候给的LOG提示说parse_url_tuple 函数既不是永久函数也不是临时函数
但是直接在HIVE中执行这句就可以用
求解决办法