“工欲善其事,必先利其器”,知识图谱作为一种特殊的图结构,自然需要专门的图数据库进行存储。
知识图谱由于其数据包含实体、属性、关系等,常见的关系型数据库诸如MySQL之类不能很好的体现数据的这些特点,因此知识图谱数据的存储一般是采用图数据库(Graph Databases)。而Neo4j是其中最为常见的图数据库。
首先在 Neo4J官网 下载 Neo4J Server。
在Mac或者Linux中,解压下载好的 Neo4J 包即可
需要先安装好 jdk
为了后续使用 neo4j 相关命令的方便,可以添加环境变量:
在 .bashrc
或 .zshrc
中添加:
export NEO4J_HOME=你安装的neo4j目录/neo4j-community-x.x.x
export PATH=$PATH:$NEO4J_HOME/bin
source
该文件,使之生效
启动:
neo4j start
停止服务:
neo4j stop
windows系统下载好neo4j和jdk 1.8.0后,输入以下命令启动后neo4j
neo4j.bat console
Neo4J提供了一个用户友好的 Web 界面,可以进行各项配置、写入、查询等操作,并且提供了可视化功能。类似ElasticSearch一样,我个人非常喜欢这种开箱即用的设计。
打开浏览器,输入http://127.0.0.1:7474/browser/,如下图所示,界面最上方就是交互的输入框。
一开始的默认用户名和密码均为:neo4j
,首次输入需要改密码,一定要牢记。
这个案例的节点主要包括人物和城市两类,人物和人物之间有朋友、夫妻等关系,人物和城市之间有出生地的关系。特别鸣谢知乎@异尘手把手教你快速入门知识图谱 - Neo4J教程
以下操作均使用 Cypher 语言,它是声明性图形查询语言,简而言之,这种语句本身看起来就很像图形。
MATCH (n) RETURN n LIMIT 25 // 查询操作
MATCH (n) DETACH DELETE n // 删除操作
这里,MATCH是匹配操作,而小括号()代表一个节点node(可理解为括号类似一个圆形),括号里面的n为标识符。
CREATE (n:Person {name:'John'}) RETURN n
注:
CREATE是创建操作,Person是标签,代表节点的类型。
花括号{}代表节点的属性,属性类似Python的字典。
这条语句的含义就是创建一个标签为Person的节点,该节点具有一个name属性,属性值是John。
CREATE (n:Person {name:'Sally'}) RETURN n
CREATE (n:Person {name:'Steve'}) RETURN n
CREATE (n:Person {name:'Mike'}) RETURN n
CREATE (n:Person {name:'Liz'}) RETURN n
CREATE (n:Person {name:'Shawn'}) RETURN n
如图所示,6个人物节点创建成功
CREATE (n:Location {city:'Miami', state:'FL'})
CREATE (n:Location {city:'Boston', state:'MA'})
CREATE (n:Location {city:'Lynn', state:'MA'})
CREATE (n:Location {city:'Portland', state:'ME'})
CREATE (n:Location {city:'San Francisco', state:'CA'})
可以看到,节点类型为Location,属性包括city和state。
如图所示,共有6个人物节点、5个地区节点,Neo4J贴心地使用不用的颜色来表示不同类型的节点。
MATCH (a:Person {name:'Liz'}),
(b:Person {name:'Mike'})
MERGE (a)-[:FRIENDS]->(b)
注:
方括号[]即为关系,FRIENDS为关系的类型。
注意这里的箭头–>是有方向的,表示是从a到b的关系。 这样,Liz和Mike之间建立了FRIENDS关系。
MATCH (a:Person {name:'Shawn'}),
(b:Person {name:'Sally'})
MERGE (a)-[:FRIENDS {since:2001}]->(b)
MATCH (a:Person {name:'Shawn'}), (b:Person {name:'John'}) MERGE (a)-[:FRIENDS {since:2012}]->(b)
MATCH (a:Person {name:'Mike'}), (b:Person {name:'Shawn'}) MERGE (a)-[:FRIENDS {since:2006}]->(b)
MATCH (a:Person {name:'Sally'}), (b:Person {name:'Steve'}) MERGE (a)-[:FRIENDS {since:2006}]->(b)
MATCH (a:Person {name:'Liz'}), (b:Person {name:'John'}) MERGE (a)-[:MARRIED {since:1998}]->(b)
这样,图谱就已经建立好了:
MATCH (a:Person {name:'John'}), (b:Location {city:'Boston'}) MERGE (a)-[:BORN_IN {year:1978}]->(b)
MATCH (a:Person {name:'Liz'}), (b:Location {city:'Boston'}) MERGE (a)-[:BORN_IN {year:1981}]->(b)
MATCH (a:Person {name:'Mike'}), (b:Location {city:'San Francisco'}) MERGE (a)-[:BORN_IN {year:1960}]->(b)
MATCH (a:Person {name:'Shawn'}), (b:Location {city:'Miami'}) MERGE (a)-[:BORN_IN {year:1960}]->(b)
MATCH (a:Person {name:'Steve'}), (b:Location {city:'Lynn'}) MERGE (a)-[:BORN_IN {year:1970}]->(b)
这里的关系是BORN_IN,表示出生地,同样有一个属性,表示出生年份。
如图 18 ,在人物节点和地区节点之间,人物出生地关系已建立好。
CREATE (a:Person {name:'Todd'})-[r:FRIENDS]->(b:Person {name:'Carlos'})
最终该图谱如下图所示:
MATCH (a:Person)-[:BORN_IN]->(b:Location {city:'Boston'}) RETURN a,b
结果如图:
MATCH (a)--() RETURN a
结果如图:
MATCH (a)-[r]->() RETURN a.name, type(r)
结果如图21:
MATCH (a)-[r]->() RETURN a.name, type(r)
结果如图:
MATCH (n)-[:MARRIED]-() RETURN n
结果如图:
MATCH (a:Person {name:'Mike'})-[r1:FRIENDS]-()-[r2:FRIENDS]-(friend_of_a_friend) RETURN friend_of_a_friend.name AS fofName
返回Mike的朋友的朋友,结果如图:
MATCH (a:Person {name:'Liz'}) SET a.age=34
MATCH (a:Person {name:'Shawn'}) SET a.age=32
MATCH (a:Person {name:'John'}) SET a.age=44
MATCH (a:Person {name:'Mike'}) SET a.age=25
这里,SET表示修改操作
MATCH (a:Person {name:'Mike'}) SET a.test='test'
MATCH (a:Person {name:'Mike'}) REMOVE a.test
删除属性操作主要通过REMOVE
3. 删除节点
MATCH (a:Location {city:'Portland'}) DELETE a
删除节点操作是DELETE
4. 删除有关系的节点
MATCH (a:Person {name:'Todd'})-[rel]-(b:Person) DELETE a,b,rel
本节主要摘自:https://github.com/datawhalechina/team-learning-nlp/blob/master/KnowledgeGraph_Basic/task01.md#%E4%BA%94neo4j-%E5%AE%9E%E6%88%98
文中做了略微的修改。
注:
;
,或者去除每一句中的变量名官方称之为:Python neo4j driver
Example 1. Acquire the driver
To find the latest stable version of the Python Driver, visit https://pypi.org/project/neo4j/
To find a list of all available releases, visit https://pypi.org/simple/neo4j/
To install the latest stable version of the Python Driver:
pip install neo4j
It is also an option to install a certain version of the driver.
Example 2. Installation with Python
The following is the syntax for installing a certain version of the Python Driver:
pip install neo4j==$PYTHON_DRIVER_VERSION
In the following example we are installing Python Driver version 4.2.1.
pip install neo4j==4.2.1
Example 3. Installation with Python, get Python Driver prerelease
In the following example we get the latest prerelease version:
pip install neo4 --pre
The release notes for the Python Driver are available here
Ref: https://neo4j.com/docs/python-manual/current/get-started/#python-driver-get-started-installation
# step 1:导入 Neo4j 驱动包
from neo4j import GraphDatabase
# step 2:连接 Neo4j 图数据库
driver = GraphDatabase.driver("bolt://localhost:7687", auth=("neo4j", "password"))
# 添加 关系 函数
def add_friend(tx, name, friend_name):
tx.run("MERGE (a:Person {name: $name}) "
"MERGE (a)-[:KNOWS]->(friend:Person {name: $friend_name})",
name=name, friend_name=friend_name)
# 定义 关系函数
def print_friends(tx, name):
for record in tx.run("MATCH (a:Person)-[:KNOWS]->(friend) WHERE a.name = $name "
"RETURN friend.name ORDER BY friend.name", name=name):
print(record["friend.name"])
# step 3:运行
with driver.session() as session:
session.write_transaction(add_friend, "Arthur", "Guinevere")
session.write_transaction(add_friend, "Arthur", "Lancelot")
session.write_transaction(add_friend, "Arthur", "Merlin")
session.read_transaction(print_friends, "Arthur")
如果 http://localhost:7474/browser/ 还开着,运行 MATCH (n) RETURN n LIMIT 25
可以查看刚刚插入的 4 个节点以及它们之间的 3 条关系(这里只截取了整个图数据库的一部分):
需要先使用 pip install py2neo
安装。
# step 1:导包
from py2neo import Graph, Node, Relationship
# step 2:构建图
g = Graph("bolt://neo4j:your_password@localhost:7687")
# step 3:创建节点
tx = g.begin()
a = Node("Person", name="Alice")
tx.create(a)
b = Node("Person", name="Bob")
# step 4:创建边
ab = Relationship(a, "KNOWS", b)
# step 5:运行
tx.create(ab)
tx.commit()
效果图:
py2neo 模块符合 Python 的习惯,写着感觉顺畅,其实可以完全不会CQL也能写。
step 2 中的参数的具体用法,参考:https://py2neo.org/2020.1/database/index.html#uris
PS: 说到这里,我想到了另一个包:NetworkX。这是一个 Python 包,可用于创建、操作和研究复杂网络的结构、动态变化和作用。它和 py2neo 一样易于操作,不过它是存在于内存中的图,可以进行一些高性能的操作。
NetworkX is a Python package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks. —— https://networkx.org/
LOAD CSV
is great for importing small- or medium-sized data (up to 10M records). For datasets larger than this, you can use the command line bulk importer. The neo4j-admin import
tool allows you to import CSV data to an empty database by specifying node files and relationship files.
Suppose you want to this tool it to import order data into Neo4j. Here are the CSV files. Notice that some of the include headers and some will have separate header files. If you want to perform the import, you place them in the import folder for your Neo4j instance.
customerId:ID(Customer), name
23, Delicatessen Inc
42, Delicious Bakery
productId:ID(Product), name, price, :LABEL
11,Chocolate,10,Product;Food
orderId:ID(Order),date,total,customerId:IGNORE
:END_ID(Order),date:IGNORE,total:IGNORE,:START_ID(Customer)
1041,2020-05-10,130,23
1042,2020-05-12,20,42
:START_ID(Order),amount,price,:END_ID(Product)
1041,13,130,11
1042,2,20,11
The tool is located in
and you run it in a terminal window where you have navigated to the import folder for your Neo4j instance.
Here is an example of importing the above CSV files in Neo4j 4.x. You must specify the name of the database. In this case we specify orders.
创建上面的几个 csv 文件后,运行以下命令:(注意,目前新版本的 Neo4j 只需要指定数据库的名称即可)
neo4j-admin import --database orders \
--nodes=Customer=customers.csv \
--nodes=products.csv \
--nodes=Order="orders_header.csv,orders1.csv,orders2.csv" \
--relationships=CONTAINS=order_details.csv \
--relationships=ORDERED="customer_orders_header.csv,orders1.csv,orders2.csv" \
--trim-strings=true
结果:
IMPORT DONE in 794ms.
Imported:
5 nodes
4 relationships
17 properties
Peak memory usage: 1.004GiB
Ref: https://neo4j.com/developer/guide-import-csv/#batch-importer
参考:https://neo4j.com/docs/cypher-manual/4.2/clauses/load-csv/