Code Generation

Explicit code and no magic - this is one of the core principles of Monstera. The goal was to achieve type-safety and utilize compile-time checks without reflection, but eliminate human mistakes from vast boilerplate code at the same time. That is why all boilerplate code is generated by the framework.

Protobufs were chosen as the technology for underlying data structures:

• Marshaling/unmarshaling comes out of the box.
• Types can be easily reused and composed with each other.
• Types can be reused between storage and core API.
• Stubs and adapters are type-safe and straightforward.

monstera.yaml defines all application cores and their operations. The following command runs codegen (from the same directory where the YAML file is located):

go tool github.com/evrblk/monstera/cmd/monstera code generate

Codegen produces three files into the same directory:

• api.go with interfaces for application cores and stubs.
• adapters.go with adapters to application cores, that turn binary blobs into routable requests.
• stubs.go with service stubs, that turn requests into binary blobs and route them to the correct application core.

Lets take a look at an example monstera.yaml:

go_code:
  output_package: github.com/evrblk/monstera-example/tinyurl
  corepb_package: github.com/evrblk/monstera-example/tinyurl/corepb
cores:
  - name: ShortUrls
    reads:
      - method: GetShortUrl
        sharded: true
        allow_read_from_followers: true
      - method: ListShortUrls
        sharded: true
    updates:
      - method: CreateShortUrl
        sharded: true
    update_request_proto: UpdateRequest
    update_response_proto: UpdateResponse
    read_request_proto: ReadRequest
    read_response_proto: ReadResponse

  - name: Users
    reads:
      - method: GetUser
        sharded: true
    updates:
      - method: CreateUser
        sharded: true
    update_request_proto: UpdateRequest
    update_response_proto: UpdateResponse
    read_request_proto: ReadRequest
    read_response_proto: ReadResponse
stubs:
  - name: TinyUrlService
    cores:
      - ShortUrls
      - Users

This config will generate:

• TinyUrlServiceCoreApi - interface for the stub TinyUrlService which consists of methods from ShortUrls and Users application cores. There are two generated implementations of this interface:
  • TinyUrlServiceCoreApiMonsteraStub - the stub that wraps Monstera Client. All the routing is done there.
  • TinyUrlServiceCoreApiStandaloneStub - standalone stub. See Standalone Application for more details.
• ShortUrlsCoreAdapter and UsersCoreAdapter - adapters for corresponding application cores ShortUrls and Users.
• ShortUrlsCoreApi and UsersCoreApi - interfaces for application cores.
• TinyUrlServiceMonsteraShardKeyCalculator - interface for a shard key extractor for all sharded methods of TinyUrlService.

Currently, Monstera codegen relies on several conventions in order to make it work with your code:

• Methods of application core must have corresponding *Response and *Request objects in go_code.corepb_package package. For example, GetShortUrl of ShortUrls core must have GetShortUrlRequest and GetShortUrlResponse proto messages in github.com/evrblk/monstera-example/tinyurl/corepb.
• *Response and *Request objects must be included into oneof of corresponding high level containers update_request_proto, update_response_proto, etc. For example, CreateShortUrlRequest must be included into UpdateRequest, CreateShortUrlkResponse must be included into UpdateResponse.
• Response containers read_response_proto and update_response_proto objects have com.evrblk.monstera.monsterax.Error error = 1; field.

You should define *Response and *Request objects for each method. This is conceptually similar to gRPC service declaration. You also should define high level containers UpdateRequest, UpdateResponse, ReadRequest, and ReadResponse. The reason why high level containers are not generated is because of protobuf field tags: they must be consistent and never change. Proto file is the right place for that, otherwise they would need to be assigned in the YAML file.

Here is an example proto file with all definitions necessary for the YAML config from above:

syntax = "proto3";

option go_package = "github.com/evrblk/monstera-example/tinyurl/corepb";
package com.evrblk.monstera_example.tinyurl.corepb;

message GetShortUrlRequest {/*...*/}
message GetShortUrlResponse {/*...*/}
message ListShortUrlsRequest {/*...*/}
message ListShortUrlsResponse {/*...*/}
message CreateShortUrlRequest {/*...*/}
message CreateShortUrlResponse {/*...*/}
message GetUserRequest {/*...*/}
message GetUserResponse {/*...*/}
message CreateUserRequest {/*...*/}
message CreateUserResponse {/*...*/}

message ReadRequest {
  reserved 1;

  oneof request {
    GetShortUrlRequest get_short_url_request = 2;
    ListShortUrlsRequest list_short_urls_request = 3;
    GetUserRequest get_user_request = 4;
  }
}

message ReadResponse {
  com.evrblk.monstera.monsterax.Error error = 1;

  oneof response {
    GetShortUrlResponse get_short_url_response = 2;
    ListShortUrlsResponse list_short_urls_response = 3;
    GetUserResponse get_user_response = 4;
  }
}calculator

message UpdateRequest {
  reserved 1;

  oneof request {
    CreateUserRequest create_user_request = 2;
    CreateShortUrlRequest create_short_url_request = 3;
  }
}

message UpdateResponse {
  com.evrblk.monstera.monsterax.Error error = 1;

  oneof response {
    CreateUserResponse create_user_response = 2;
    CreateShortUrlResponse create_short_url_response = 3;
  }
}

Marshalled UpdateRequest, UpdateResponse, ReadRequest, and ReadResponse objects is what actually exchanged over Monstera client.

What is left for you to implement:

• Application cores themselves (ShortUrlsCoreApi and UsersCoreApi interfaces).
• Shard key extractor (TinyUrlServiceMonsteraShardKeyCalculator interface).

Shard key extractor interface is a simple interface that has corresponding functions for each *Request object of sharded methods from YAML file (marked with sharded: true). There are no annotations or reflection to extract shard keys from proto requests. User specifies explicitly how to extract a shard key from each request with one line of Go code.