oslo.versionedobjects 分析

openstack python oslo.versionedobjects

[toc]

oslo.versionedobjects庫提供了一個通用版本化的對象模型，它是rpc友好的，具有內(nèi)置的序列化、字段類型和遠(yuǎn)程方法調(diào)用。它可以用于在一個獨(dú)立于外部api或數(shù)據(jù)庫模式的項目內(nèi)定義數(shù)據(jù)模型，以提供跨分布式服務(wù)的升級兼容性。

一、安裝

$ pip install oslo.versionedobjects

要使用 oslo_versionedobjects.fixture，還需要安裝其他依賴：

$ pip install 'oslo.versionedobjects[fixtures]'

二、使用步驟

1. 把 oslo.versionedobjects 加入 requirements

# cinder\requirements.txt:

oslo.versionedobjects>=1.17.0 # Apache-2.0

2. 創(chuàng)建一個子目錄，起名 objects，并在里面添加python文件 base.py

<project>/objects 是實體類的存放目錄。

比如backup類:

@base.CinderObjectRegistry.register
class Backup(base.CinderPersistentObject, base.CinderObject,
             base.CinderObjectDictCompat):
    # Version 1.0: Initial version
    # Version 1.1: Add new field num_dependent_backups and extra fields
    #              is_incremental and has_dependent_backups.
    # Version 1.2: Add new field snapshot_id and data_timestamp.
    # Version 1.3: Changed 'status' field to use BackupStatusField
    # Version 1.4: Add restore_volume_id
    VERSION = '1.4'

    fields = {
        'id': fields.UUIDField(),

        'user_id': fields.StringField(),
        'project_id': fields.StringField(),

        'volume_id': fields.UUIDField(),
        'host': fields.StringField(nullable=True),
        'availability_zone': fields.StringField(nullable=True),
        'container': fields.StringField(nullable=True),
        'parent_id': fields.StringField(nullable=True),
        'status': c_fields.BackupStatusField(nullable=True),
        'fail_reason': fields.StringField(nullable=True),
        'size': fields.IntegerField(nullable=True),

        'display_name': fields.StringField(nullable=True),
        'display_description': fields.StringField(nullable=True),

        # NOTE(dulek): Metadata field is used to store any strings by backup
        # drivers, that's why it can't be DictOfStringsField.
        'service_metadata': fields.StringField(nullable=True),
        'service': fields.StringField(nullable=True),

        'object_count': fields.IntegerField(nullable=True),

        'temp_volume_id': fields.StringField(nullable=True),
        'temp_snapshot_id': fields.StringField(nullable=True),
        'num_dependent_backups': fields.IntegerField(nullable=True),
        'snapshot_id': fields.StringField(nullable=True),
        'data_timestamp': fields.DateTimeField(nullable=True),
        'restore_volume_id': fields.StringField(nullable=True),
    }

    obj_extra_fields = ['name', 'is_incremental', 'has_dependent_backups']

3. objects/base.py 里為項目創(chuàng)建一個具有項目命名空間的VersionedObject基類，繼承 oslo_versionedobjects.base.VersionedObject

必須填滿OBJ_PROJECT_NAMESPACE屬性。OBJ_SERIAL_NAMESPACE只用于向后兼容，不應(yīng)該設(shè)置在新項目中。
如：

# cinder.objects.base.CinderObject:

class CinderObject(base.VersionedObject):
    # NOTE(thangp): OBJ_PROJECT_NAMESPACE needs to be set so that nova,
    # cinder, and other objects can exist on the same bus and be distinguished
    # from one another.
    OBJ_PROJECT_NAMESPACE = 'cinder'
    <!--省略-->

4. 創(chuàng)建一個持久化對象類PersistentObject

這是一個持久對象的mixin類，定義重復(fù)的字段，如created_at、updated_at。Mixin是有部分或者全部實現(xiàn)的接口，其主要作用是代碼復(fù)用，用于多繼承，具體可以參考博客。
在這個類可以定義一些用于數(shù)據(jù)庫持久化的常用字段，比如created_at、updated_at等，還可以定義一些持久化常用的方法，比如get_by_id()、refresh()、exists()。這些屬性和方法可以被其他子類繼承使用。
比如：

# cinder.objects.base.CinderPersistentObject:

class CinderPersistentObject(object):
    """Mixin class for Persistent objects.

    This adds the fields that we use in common for all persistent objects.
    """
    OPTIONAL_FIELDS = []

    Not = db.Not
    Case = db.Case

    fields = {
        'created_at': fields.DateTimeField(nullable=True),
        'updated_at': fields.DateTimeField(nullable=True),
        'deleted_at': fields.DateTimeField(nullable=True),
        'deleted': fields.BooleanField(default=False,
                                       nullable=True),
    }
    
    @classmethod
    def exists(cls, context, id_):
        return db.resource_exists(context, cls.model, id_)
    <!--省略-->

Implement objects and place them in objects/*.py?

5.添加實體類文件（這段看不懂?。。。?/h2>

Objects classes should be created for all resources/objects passed via RPC as IDs or dicts in order to:

• spare the database (or other resource) from extra calls
• pass objects instead of dicts, which are tagged with their version
• handle all object versions in one place (the obj_make_compatible method)

To make sure all objects are accessible at all times, you should import them in init.py in the objects/ directory.

6. 新建objects/fields.py

在fields.py，可以創(chuàng)建一些繼承oslo_versionedobjects.field.Field類的Field類，在里面定義一些常量字段，也可重寫 from_primitive 和 to_primitive 兩個方法。
子類化oslo_versionedobjects.fields.AutoTypedField，可以將多個屬性堆疊在一起，確保即使是嵌套的數(shù)據(jù)結(jié)構(gòu)也得到驗證。

如：

# cinder\objects\fields.py

from oslo_versionedobjects import fields

BaseEnumField = fields.BaseEnumField
Enum = fields.Enum
Field = fields.Field
FieldType = fields.FieldType

class BaseCinderEnum(Enum):
    def __init__(self):
        super(BaseCinderEnum, self).__init__(valid_values=self.__class__.ALL)

class BackupStatus(BaseCinderEnum):
    ERROR = 'error'
    ERROR_DELETING = 'error_deleting'
    CREATING = 'creating'
    AVAILABLE = 'available'
    DELETING = 'deleting'
    DELETED = 'deleted'
    RESTORING = 'restoring'

    ALL = (ERROR, ERROR_DELETING, CREATING, AVAILABLE, DELETING, DELETED,
           RESTORING)

7. 創(chuàng)建對象注冊類，用于注冊所有實體類

繼承oslo_versionedobjects.base.VersionedObjectRegistry
這是所有對象被注冊的地方。所有對象類都應(yīng)該通過oslo_versionedobjects.base.ObjectRegistry裝飾器被注冊。
如：

定義Cinder對象注冊裝飾器類：

# cinder\objects\base.py

class CinderObjectRegistry(base.VersionedObjectRegistry):
    def registration_hook(self, cls, index):
        """Hook called when registering a class.

        This method takes care of adding the class to cinder.objects namespace.

        Should registering class have a method called cinder_ovo_cls_init it
        will be called to support class initialization.  This is convenient
        for all persistent classes that need to register their models.
        """
        setattr(objects, cls.obj_name(), cls)

        # If registering class has a callable initialization method, call it.
        if callable(getattr(cls, 'cinder_ovo_cls_init', None)):
            cls.cinder_ovo_cls_init()

使用的時候只需在實體類上面添加一行@base.CinderObjectRegistry.register即可。如：

# cinder\objects\cluster.py

@base.CinderObjectRegistry.register
class Cluster(base.CinderPersistentObject, base.CinderObject,
              base.CinderComparableObject):

# cinder\objects\consistencygroup.py

@base.CinderObjectRegistry.register
class ConsistencyGroup(base.CinderPersistentObject, base.CinderObject,
                       base.CinderObjectDictCompat, base.ClusteredObject):              

8.創(chuàng)建和連接序列化器

oslo_versionedobjects.base.VersionedObjectSerializer

為了用于RPC傳輸對象，我們需要創(chuàng)建oslo_versionedobjects.base.VersionedObjectSerializer的子類，并通過設(shè)置OBJ_BASE_CLASS屬性預(yù)定義對象類型。
如：

class CinderObjectSerializer(base.VersionedObjectSerializer):
    OBJ_BASE_CLASS = CinderObject
    <!--省略-->

然后將序列號器連接至oslo_messaging：

# cinder\rpc.py

def get_client(target, version_cap=None, serializer=None):
    assert TRANSPORT is not None
    serializer = RequestContextSerializer(serializer)
    return messaging.RPCClient(TRANSPORT,
                               target,
                               version_cap=version_cap,
                               serializer=serializer)


def get_server(target, endpoints, serializer=None):
    assert TRANSPORT is not None
    serializer = RequestContextSerializer(serializer)
    return messaging.get_rpc_server(TRANSPORT,
                                    target,
                                    endpoints,
                                    executor='eventlet',
                                    serializer=serializer)

9.實現(xiàn)間接的api

oslo_versionedobjects.base.VersionedObjectIndirectionAPI

對于這個類，官網(wǎng)是這么解釋的：

oslo.versionedobjects supports remotable method calls. These are calls of the object methods and classmethods which can be executed locally or remotely depending on the configuration. Setting the indirection_api as a property of an object relays the calls to decorated methods through the defined RPC API. The attachment of the indirection_api should be handled by configuration at startup time.

Second function of the indirection API is backporting. When the object serializer attempts to deserialize an object with a future version, not supported by the current instance, it calls the object_backport method in an attempt to backport the object to a version which can then be handled as normal.

翻譯過來就是：
oslo.versionedobjects支持遠(yuǎn)程方法調(diào)用。這些是對象方法和類方法，它們可以在本地執(zhí)行，也可以根據(jù)配置遠(yuǎn)程執(zhí)行。將indirection_api設(shè)置為對象的屬性，通過定義的RPC API將調(diào)用傳遞給裝飾方法。indirection_api的附件在啟動時應(yīng)該由配置來處理。
間接API的第二個功能是反向移植。當(dāng)對象序列化器試圖用將來的版本來反序列化一個對象時，它不支持當(dāng)前實例，它調(diào)用object_backport方法，試圖將對象反向移植到一個可以正常處理的版本上。

但是我看不懂它到底想表示啥，又沒有范例。cinder好像也沒用用到。。。

三、官方示例

# -*- coding: utf-8 -*-

from datetime import datetime

from oslo_versionedobjects import base
from oslo_versionedobjects import fields as obj_fields


# INTRO: This example shows how a object (a plain-old-python-object) with
# some associated fields can be used, and some of its built-in methods can
# be used to convert that object into a primitive and back again (as well
# as determine simple changes on it.


# Ensure that we always register our object with an object registry,
# so that it can be deserialized from its primitive form.

# 燈泡類，繼承自VersionedObject
@base.VersionedObjectRegistry.register
class IOTLightbulb(base.VersionedObject):
    """Simple light bulb class with some data about it."""

    VERSION = '1.0'  # Initial version

    #: Namespace these examples will use.
    OBJ_PROJECT_NAMESPACE = 'versionedobjects.examples'

    # 必填的參數(shù)
    #: Required fields this object **must** declare.
    fields = {
        'serial': obj_fields.StringField(),
        'manufactured_on': obj_fields.DateTimeField(),
    }


# 初始化一個bulb對象
bulb = IOTLightbulb(serial='abc-123', manufactured_on=datetime.now())
print(bulb.VERSION)
print("The __str__() output of this new object: %s" % bulb)
print("The 'serial' field of the object: %s" % bulb.serial)
# 把對象的屬性轉(zhuǎn)換成原始表單格式（看起來就是json）
bulb_prim = bulb.obj_to_primitive()
print("Primitive representation of this object: %s" % bulb_prim)

# 從原始狀態(tài)還原bulb_prim到bulb對象
bulb = IOTLightbulb.obj_from_primitive(bulb_prim)

bulb.obj_reset_changes()
print("The __str__() output of this new (reconstructed)"
      " object: %s" % bulb)

# 修改bulb的屬性值，通過obj_what_changed()函數(shù)可以查出是哪個屬性做了修改。
bulb.serial = 'abc-124'
print("After serial number change, the set of fields that"
      " have been mutated is: %s" % bulb.obj_what_changed())

運(yùn)行結(jié)果：

The __str__() output of this new object: IOTLightbulb(manufactured_on=2017-08-04T17:41:43Z,serial='abc-123')
The 'serial' field of the object: abc-123
Primitive representation of this object: {'versioned_object.version': '1.0', 'versioned_object.changes': ['serial', 'manufactured_on'], 'versioned_object.name': 'IOTLightbulb', 'versioned_object.data': {'serial': u'abc-123', 'manufactured_on': '2017-08-04T17:41:43Z'}, 'versioned_object.namespace': 'versionedobjects.examples'}
The __str__() output of this new (reconstructed) object: IOTLightbulb(manufactured_on=2017-08-04T17:41:43Z,serial='abc-123')
After serial number change, the set of fields that have been mutated is: set(['serial'])

Process finished with exit code 0