Protocol context manager

+1  

In big microservice based systems or systems with lots of protocols, you often need to make a change that ripples out through the entire system. If you have X services you send a message between them you have X message schemas. Let's manage the schemas centrally

YAML 想法

Adding a field to a message should be a simple change in a central system. The rest of the system should just adapt to the new field.

In traditional microservice architectures there's either a monorepo or a repository per microservice and each service has to be changed to add the new field.

In this design each service retrieves its schema from a central source and every area where messages are constructed there is a context area. This context area contains all known state at that point in time.

Someone can move a piece of context to the protocol schema centrally. So adding a new column to a database or to a protocol message is as simple as changing it in one place.

chronological,


(別通知) (可選) 請,登錄

對我來說,它看起來像一些“docker registry”(~“schema registry”)。所以,我請一位專門從事 DevOps 的朋友對此發表評論。他的想法是:

“有道理,但是數據庫或文件,另一個狀態功能應該只有所有者,如果各種服務使用 1 db,則意味着您將失去一致性。就架構而言,是的,這是有道理的。但是服務的所有者使用不同API 的版本。一旦你在 API 上發佈了一些新的變化,你必須增加 API 版本,比如 /api/v2/blabla。我認爲 Protobuf 或 graphql可以部分解決這個問題。”

我:“所以,你的意思是,可以提出一點,協議緩衝區和 API 版本控制使得消息模式的中央管理不再必要?”

他:“所有現代 API 提供商都使用版本控制……好吧,字段會自動出現,但如果應用程序不知道它是什麼,那麼該字段就沒有用了。”

// 有人可以將一段上下文集中移動到協議模式中。因此,將新列添加到數據庫或協議消息就像在一個地方更改它一樣簡單。

就我個人而言,我喜歡獨立於微服務開發數據模式的想法,因爲這樣每個人都可以成爲系統本體模型開發的一部分,這永遠不會只以一個微服務結束。

For me, it looked like some "docker registry" (~"schema registry"). So, I asked a friend specialized in devops to comment on this. His thoughts were:

"Makes sense, however a db or files, another state feature are supposed to have only owner, if various services use 1 db, it means you will loose consistency. In terms of schema yes, it makes sense. However owner of service use different versions for API. Once you release some new changes on API, you have to increase API version, like /api/v2/blabla.I think Protobuf or graphql can solve this problem partly."

Me: "So, you mean, a point could be made, that protocol buffers, and API versioning make the central management of messaging schemas not necessary?"

Him: "All modern API providers use versioning... Ok, field appears automatically, but if application is not aware what it is, the field is unuseful."

// Someone can move a piece of context to the protocol schema centrally. So adding a new column to a database or to a protocol message is as simple as changing it in one place.

Personally, I like the idea of developing schemas of data independently of microservices, because then everyone can be part of the development of ontological model of the system, which never ends with just one microservice.


許多服務在有效負載中傳遞消息或存儲來自其他服務的數據。有時,有問題的服務並不特別關心特定的領域,但另一個服務可能會在鏈的下游。

例如,在我使用的事件驅動微服務架構中,我們讓 MongoDB 存儲來自先前服務的數據的緩存有效負載,以防止此後每個服務都獲取它。它還可以保持數據同步。

模式在 RabbitMQ 系統中使用,但通常以特殊方式進行管理。也就是說,您必須保持同步的微服務網絡的每個微服務中都有一個 Java 類。這是一場噩夢。

手動版本控制是一種解決方案,我只是認爲它可以自動完成。在架構管理 UI 中按下新版本按鈕並添加新字段。並勾選該字段適用於每種消息類型的複選框。

如果我們想要像 DAO 這樣自治的自我管理組織,則需要就協議格式達成一些協議。

Many services pass on messages or store data from other services within the payload. Sometimes the service in question doesn't care about particular fields particularly but another service might further down the chain.

For example in event driven microservice architecture I have used we had MongoDB store cached payload of data from a previous service to preclude having every service thereafter fetching it. Also it keeps data in sync.

Schemas are used in RabbitMQ systems but are often managed in an ad hoc way. That is you have a Java class in each microservice of the network of microservices that you have to keep in sync. It's a nightmare.

Versioning manually is a solution I just think it could be done automatically. Press a new version button in a schema management UI and add the new fields. And tick check boxes where the field applies for each message kind.

If we want autonomous self managed organisations like DAOs there will need to be some agreement on the protocol format.



    :  -- 
    : Mindey
    :  -- 
    

chronological,

模式註冊表可以在高可用性模式下運行。

orderData = db.get_order(order_id)

context.put("order", order_data)

context.put("user_id", user_id)

message = context.render("new-order-message")

在模式註冊表中查找 new-order-message 以查看它包含哪些數據

註冊表決定在呈現時出現在消息有效負載中的上下文或上下文子對象中的哪些數據。

The schema registry could be ran in high availability mode.

orderData = db.get_order(order_id)

context.put("order", order_data)

context.put("user_id", user_id)

message = context.render("new-order-message")

Where new-order-message is looked up in the schema registry to see what data it contains

The registry decides what data in context or inside context subobjects appears in the message payload when rendered.



    :  -- 
    : Mindey
    :  -- 
    

chronological,

在我基於電子郵件的社交網絡中,我預先定義了許多模式。處理各種事情的股份。

我爲每條消息編寫了示例 XML。但是在寫完這些例子後沒有使用它們。

In my email based social network I defined lots of schemas upfront. To handle shares of various things.

I wrote example XMLs of each message. But didn't use the examples after writing them.



    :  -- 
    : Mindey
    :  -- 
    

chronological,