spark中的graphframe模块

Yangstal 2020-06-21 04:49:15

# Build `tripGraph` GraphFrame
# This GraphFrame builds up on the vertices and edges based on our trips (flights)
tripGraph = GraphFrame(tripVertices, tripEdges)
print (tripGraph)

# Build `tripGraphPrime` GraphFrame
# This graphframe contains a smaller subset of data to make it easier to display motifs and subgraphs (below)
tripEdgesPrime = departureDelays_geo.select("tripid", "delay", "src", "dst")
tripGraphPrime = GraphFrame(tripVertices, tripEdgesPrime)

error
java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)[Ljava/lang/Object;
---------------------------------------------------------------------------
Py4JJavaError Traceback (most recent call last)
<command-3237926837849918> in <module>
1 # Build `tripGraph` GraphFrame
2 # This GraphFrame builds up on the vertices and edges based on our trips (flights)
----> 3 tripGraph = GraphFrame(tripVertices, tripEdges)
4 print (tripGraph)
5

/local_disk0/spark-bed27070-0715-4fbc-9995-65337586a164/userFiles-9ef7087e-7659-49df-a57f-5770515b198c/addedFile506950925074357278graphframes_0_8_0_spark3_0_s_2_12-a07ea.jar/graphframes/graphframe.py in __init__(self, v, e)
87 .format(self.DST, ",".join(e.columns)))
88
---> 89 self._jvm_graph = self._jvm_gf_api.createGraph(v._jdf, e._jdf)
90
91 @property

/databricks/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in __call__(self, *args)
1255 answer = self.gateway_client.send_command(command)
1256 return_value = get_return_value(
-> 1257 answer, self.gateway_client, self.target_id, self.name)

...全文