分布式架构

OrientDB可以部署在不同的服务器上，可以使用不同的方式实现最大的性能，伸缩性和健壮性。

OrientDB使用Hazelcast Open Source project来管理集群。页面上的大多参考都链接到Hazelcast的官方文档。

介绍

主题

服务器角色

OrientDB是多master分布式架构(也叫作"master-less")，每个服务器都可以读和写。从2.1版本开始，OrientDB支持"REPLICA"角色, 这种角色的服务器只读，接受幂等的命令，例如读和查询。当服务器以"REPLICA"角色加入分布式集群时, 不会像"MASTER"节点一样拥有日志簇。

记录的创建(文档, 顶点和边)

在分布式模式下，RID有簇本地生成。如果你有类Customer和3个节点(node1, node2, node3), 你将有以下簇:

  • customer 簇id等于#15 (默认，node1)
  • customer_node2 簇id等于#16
  • customer_node3 簇id等于#17

如果你在node1上创建一个新的Customer, RID将以"customer"的簇ID: #15为前缀。同样的操作在node2和node3上面一样。

通过这种方式，RID不会碰撞，每个节点都可以成为master完成插入操作，而不会有任何冲突。

分布式事务

从1.6版本开始，OrientDB支持分布式事务。当一个事务提交时， 所有更新的记录会发送给所有的服务器，每个服务器都负责提交事务。当一个或者多个节点提交失败，quorum（读写成功的节点数量）会被校验。如果quorum被遵守，失败的节点会和成功的节点保持一致，否则所有的节点都会回滚事务。

分布式事务期间的可见性是什么样子?

During the distributed transaction, in case of rollback, there could be an amount of time when the records appear changed before they are rollbacked.

Split brain network problem

OrientDB guarantees strong consistency if it's configured to have a writeQuorum set to a value as the majority of the number of nodes. I you have 5 nodes, it's 3, but if you have 4 nodes, it's still 3 to have a majority. While writeQuorum setting can be configured at database and cluster level too, it's not suggested to set a value minor than the majority of nodes, because in case of re-merge of the 2 split networks, you'd have both network partitions with updated data and OrientDB doesn't support (yet) the merging of 2 non read-only networks. So the suggestion is to always provide a writeQuorum with a value to, at least, the majority of the nodes.

Limitations

OrientDB v2.1.x has some limitations you should notice when you work in Distributed Mode:

  • hotAlignment:true could bring the database status as inconsistent. Please set it always to 'false', the default
  • Creation of a database on multiple nodes could cause synchronization problems when clusters are automatically created. Please create the databases before to run in distributed mode
  • If an error happen during CREATE RECORD, the operation is fixed across the entire cluster, but some node could have a wrong RID upper bound (the created record, then deleted as fix operation). In this case a new database deploy operation must be executed
  • Constraints with distributed databases could cause problems because some operations are executed at 2 steps: create + update. For example in some circumstance edges could be first created, then updated, but constraints like MANDATORY and NOTNULL against fields would fail at the first step making the creation of edges not possible on distributed mode.
  • Auto-Sharding is not supported in the common meaning of Distributed Hash Table (DHT). Selecting the right shard (cluster) is up to the application. This will be addressed by next releases
  • Sharded Indexes are not supported
  • If hotAlignment=false is set, when a node re-joins the cluster (after a failure or simply unreachability) the full copy of database from a node could have no all information about the shards
  • Hot change of distributed configuration not available. This will be introduced at release 2.0 via command line and in visual way in the Workbench of the Enterprise Edition (commercial licensed)
  • Not complete merging of results for all the projections when running on sharder configuration. Some functions like AVG() doesn’t work on map/reduce