Abstracting procedures

Basic requirements

A procedure in our framework can be a module-level function or a class method. In our example above, we define a module-level function with

def hello_world(name: str) -> str:
    return 'Hello World, ' + name

and make this available for the RPC. It’s a standard python function, so, nothing special here. But there is one important aspect: the so-called type annotation name: str and -> str. This feature of the python language, introduced with python 3.5, is crucial to the DASF messaging framework. In standard python, type annotations have no impact during runtime. So you can call hello_world(["ok"]) and it would run the function (and produce an error of course, as it cannot combine "Hello World, " + ["ok"]). Only if you would use a static type checker such as mypy, it would produce an error without even prior to calling the function, as it realizes that the hello_world function accepts a string, and not a list. Within DASF however, we do use the type annotation during runtime. It is an important part in any web-based framework, that you need to verify the input. But instead of adding an additional verification system, we use the type annotations here[1]. If you want to understand the internals of the DASF messaging framework, you should familiarize yourself a bit with type annotations in python.

How the abstraction works

Our framework takes standard python functions or classes and makes them available for a remote procedure call. As a web-based framework, this means that

1. Our framework needs to be able to deserialize JSON requests

2. Our framework needs to be able to validate the requests

3. Our framework needs to be able to serialize JSON requests

Let’s take our hello_world function from above. If you would want to call this function without our framework, you’d need to implement some wrapper that deserializes the request, validates the input and serializes the output, i.e. something like

import json

def hello_world_request_wrapper(request: str):
    data = json.loads(request)  # deserialize the request
    assert isinstance(data, dict)  # make sure we get a dictionary
    assert "name" in data  # check that the name is in the data
    assert isinstance(data["name"], str)  # make sure the name is a string
    result = hello_world(name=data["name"])  # call our function
    return json.dumps(result)  # serialize and return the result

You can see that such kind of serialization and validation becomes quite verbose. And now imagine you have more complicated procedure calls then our very simple example. A hard-coded verification would add a lot of overhead to the code, just to make the procedures available for RPCs.

The abstraction in the DASF works differently and (usually) without additional code. The idea is that when you specify a function such as def hello_world(name: str) -> str: we know what we expect. Everything is here: The function, the arguments, the inputs, etc. So what we are doing is, that we take pythons built-in inspect module to analyse the function, and docstring-parser for interpreting the docstring of the procedure. Then we use this information and create a subclass of pydantics BaseModel that represents this function (or class or method).

Note

pydantic is a python package that uses pythons type annotations for runtime validation of data. Moreover, it can be used to serialize and de-serialize input based on the annotations. You can find many examples in their docs.

We recommend that you go through the overview and familiarize yourself a bit with it before continuing.

We’ll stick now to our hello_world function and look how this is abstracted for the remote procedure call. Classes are handled pretty much the same, but we’ll discuss this later in Abstracting classes for RPC.

Abstracting functions for RPC

When you call the main() function, it loads the procedures that you make available for the remote procedure call (see Specifying module members). If it encounters a function, it will call the create_model() classmethod of the demessaging.backend.function.BackendFunction() class. This method creates a subclass of the BackendFunction() using pydantics create_model function that represents our hello_world function in DASF.

In other words, you end up with

from demessaging.backend.function import BackendFunction
HelloWorldModel = BackendFunction.create_model(hello_world)

The HelloWorldModel class that has been created through this method is comparable to the following class:

from demessaging.backend.function import BackendFunction
from typing import Literal

class HelloWorldModel(BackendFunction):

    func_name: Literal["hello_world"] = "hello_world"
    name: str

It’s a class with two attributes, func_name and name.

func_name is an attribute that is always added to these kind of models. It can take exactly one value, and this is the name of the function that it serialized (in this case, "hello_world"). The second attribute, name, comes from the function that has been called. The hello_world function accepts exactly one argument, the name: str:, and so this has been added as an attribute to our model.

The create_model() classmethod does create another model, the return model. Our hello_world function returns a string and also needs to be serialized, when sent to the client stub. Our return model is a __root__ model of pydantic and comparable to

from pydantic import BaseModel

class HelloWorldReturnModel(BaseModel):

    root: str

and is accessible as HelloWorldModel.return_model.

Calling the procedure

Our framework creates an instance of the created HelloWorldModel from the request. It then executes the underlying model function via

request = HelloWorldModel(name="Peter")
result_model = request()  # calls the `hello_world` function. `result_model` is an instance of HelloWorldReturnModel

assert result_model.root == hello_world(name="Peter")

But the thing is, that we can now also create and return JSON from this:

request = HelloWorldModel.parse_json('{"name": "Peter"}')
result_model = request()
result_model.model_dump_json()  # gives 'Hello World, Peter'

Configuring the Function Model Creation

You can also change how things are converted from the python function to the pydantic model using the configure() function. Passing arguments to this function would be equivalent to setting the config parameter to the call of create_model(). You can give any keyword argument here that is available for the initialization of a FunctionConfig instance. You can, for instance, add custom validators or field_params.

Abstracting classes for RPC

Instead of using a function as described above, one can also use a method of a python class. Here is an example:

class HelloClass:

    def __init__(self, text: str ="Hello"):
        self._text = text

    def hello_world(self, name: str) -> str:
        return self._text + " World, " + name

calling hello_world("Peter") is equivalent to calling HelloClass().hello_world("Peter"). When we encounter a class in the members that you specified for the main() function (see Specifying module members), then we will not use the BackendFunction to create a new model, but instead we use the BackendClass.

The create_model() classmethod of the BackendClass uses the same procedure as described in the previous section (Abstracting functions for RPC) to abstract the __init__ method of the class. And it does an abstraction for all the methods of the class.

The pydantic representation of our class, i.e. the HelloClassModel in

from demessaging.backend.class_ import BackendClass
HelloClassModel = BackendClass.create_model(HelloClass)

is comparable to:

from demessaging.backend.class_ import BackendClass
from demessaging.backend.function import BackendFunction

class HelloClassHelloWorldModel(BackendFunction):

    func_name: Literal["hello_world"] = "hello_world"
    name: str

class HelloClassModel(BackendClass):

    class_name: Literal["HelloClass"] = "HelloClass"
    text: str = "Hello"
    function: HelloClassHelloWorldModel

and you would create an instance of this method via

request = HelloClassModel(
    text="Hello",
    function={"func_name": "hello_world", "name": "Peter"}
)

Hence, you specify the method that you want to call. Running request() now creates an instance of HelloClass and runs it’s hello_world method, i.e.

response = HelloClassModel()
response.root == HelloClass(text="Hello").hello_world(name="Peter")

Configuring the Class Model Creation

You can configure classes as you can configure functions, just decorate your class with the configure() function and pass the parameters for the ClassConfig. You can, for instance, use the methods parameter to specify what methods will be available for the RPC. You can also decorate any method of the class with the configure() function as you do with normal functions.

---

[1]

nevertheless, you can also add your own validators using the configure() function