Show HN: Lockval Engine – A distributed backend KV engine that can run scripts | Hacker News Reader

Show HN: Lockval Engine – A distributed backend KV engine that can run scripts

lockval.com

44 pointsvanishs20 comments

However, these codes and frameworks have always had problems such as very complicated APIs, incomplete or non-existent ACID, multiple calls to RPC, and non-real-time full or semi-full data landing.

So, we created Lockval Engine. An easy-to-use, ACID, distributed, DOP, multi-language support backend engine.

Here are some online demos that will give you a quicker understanding of Lockval Engine:

[APIdemo] https://apidemo.lockval.com/

[playground] https://playground.lockval.com/?lang=JavaScript&code=97aa205...

Open articleView on HN

Since I used HN for the first time, I didn’t realize that if I filled in the url, the text content would not be displayed, so I reposted what I wrote before:

I've been developing game backends for several years, using and designing a lot of code. They all about calculations, synchronization with front-end data, and database data, and try to ensure consistency, atomicity, and high response, etc. However, these codes and frameworks have always had problems such as very complicated APIs, incomplete or non-existent ACID, multiple calls to RPC, and non-real-time full or semi-full data landing.

So, we created Lockval Engine. An easy-to-use, ACID, distributed, DOP, multi-language support backend engine.

Here are some online demos that will give you a quicker understanding of Lockval Engine:

[play JS] https://playground.lockval.com/?lang=JavaScript&code=63fb83b...

[play Lua] https://playground.lockval.com/?lang=Lua&code=dfda230f6ae280...

[play Python] https://playground.lockval.com/?lang=Python&code=4c6db428c37...

[APIdemo] https://apidemo.lockval.com/

[website] https://lockval.com/

> that can simulate the rules of the world in your imagination

What does this mean?

It means: it can achieve many functions that were previously impossible or not easy to achieve due to the limitations of the framework.

I have somehow feeling that this does not answer the original question. Could you give some examples of impossible or not easy tasks this achieves and other solutions don’t?

Take the recently popular actor model as an example. Each player is an actor, which can not only ensure the simplicity of development, but also ensure the atomicity of the player's data in one operation. But if a player-to-player transaction system is designed, it is difficult for this actor model to achieve the atomicity of this transaction.

But Lockval Engine can easily do all this. Because Lockval Engine is data-oriented programming. It provides a pair of APIs: GetAndLock and PutAndUnlock. You lock the data when you fetch it, and unlock it when you complete the modification.

And this pair of APIs will also synchronize the modified data to the front end. In this way, you don't need to worry about how to send data to the front end when designing functions. Other frameworks or codes need to complete this function by themselves.

Other advantages include but are not limited to: In this distributed architecture, the atomicity of hot update code and configuration. It won't happen that half the system is an old script and half is a new script.

Can you elaborate on an example with games that have some form of "fog of war"? Or rather only some state is seen by X players?

In Lockval Engine, a player data pair is on UID. A map block data can also be mapped to a UID. Or an observation point data can also correspond to a UID.

Lockval Engine provides a Watch function. Players can watch the UID of a certain map block or the UID of a certain observation point, so that a function similar to the fog of war can be realized.

This demo can be viewed in "Prepare a globalChat struct" and "send A message to globalChat" on the apidemo page. This demo can evolve a variety of similar functions.

There's some cool ideas here. From what I understand, you basically implement a few endpoints in a serverless-esque fashion, which have access to methods to store data, and the engine will keep the client updated and handle consistency?

I can definitely see this as something to bootstrap a backend quickly. It could serve as a nice in-between of Firebase/Supabase and a fully fledged server.

It took me a long time to grok the docs though. It would be great if there were more simple examples. The landing page refers to a lot of the nitty-gritty details, including method names and services in the architecture, but imo that's getting into the weeds before explaining the value prop. I also had a hard time figuring out the apidemo page on my phone screen.

First of all, thank you very much for your feedback.

Yes, as you said, gw and api in Lockval Engine implement stateless services, db implements ACID for front-end and back-end data, and trigger provides calling methods outside the engine.

So this is a complete closed-loop backend engine. It provides the most basic computing and data synchronization capabilities, and other functions can also be provided to upper-level scripts in the form of plug-ins

Sorry for the very lacking and messy documentation, I'll try to get them out of there as soon as I can. I made apidemo and playground pages, I think these two pages can help you understand how Lockval Engine is used in a more intuitive way. In addition, these two pages can be downloaded locally to build the experience yourself.

I'm also sorry that none of these pages are optimized for mobile, web development is really not my strong point.

Does Starlark have Python libraries like requests, pyquery, libpq, SQLite client, etc?

The Go angle is certainly intriguing.

For safety, the script does not provide any API that harms your environment. But you can use the Go plugin to write the API you need and provide it to the script.

The solution can be found in "Use api.*.so for more functionality" in [apidemo](https://apidemo.lockval.com/)

Go plugin code: https://github.com/lockval/api.so

> For safety, the script does not provide any API that harms your environment.

how do you ensure safety? Can the scripts make network or IO calls?

Also, how do you actually run the scripts? in some isolated docker container / VM?

The script is running in the VM on the api service and does not provide any IO functions.

Just like the functions provided by the playground page, users can fill in any code, so I must ensure that it cannot write functions that can harm the server.

However, someone wants to connect to sql, mongodb, local files, http, etc. Even if it is the same http request function, everyone's needs may be different, so the Go plugin function mentioned above allows users to develop the functions they want. In this way, someone will make a more useful mysql plug-in, someone will make a mongodb plug-in, and so on. If you can't find the function you want, you can also develop one yourself.

> The script is running in the VM on the api service and does not provide any IO functions.

Thank you for answering! but I am still not sure how can you prevent someone importing io lib in go or js and writing to a file.

> Just like the functions provided by the playground page, users can fill in any code, so I must ensure that it cannot write functions that can harm the server.

can you tell me more on how are you ensuring this?

python, Lua and JS These functions are implemented by these packages:

github.com/dop251/goja

github.com/yuin/gopher-lua

go.starlark.net

Go scripts (programs) are not subject to any restrictions. So there is no Go script support in the playground(debug)

Thank you so much! This explains how they are run. I will explore these repositories.

thank you for discussing with me

In addition, Lockval Engine includes db and several supporting APIs.

Lockval Engine is developed around the concept of data-oriented programming. Therefore, it is generally not recommended to use an external database.

The db in Lockval Engine is a bit like redis, which provides hot data in memory and saves changing cold data in real time

Regarding the location of db in Lockval Engine, you can view the architecture diagram on this page: https://lockval.com/docs/

Is the engine open source? The main lockval repo on GitHub just has docs.

Lockval Engine is not yet open source software, and the code and documentation of some peripheral tools are on the github homepage.

I have been thinking about the most suitable business model for Lockval Engine, such as finding suitable funders.

This is not an issue that needs to be considered recently. Lockval Engine still has many things to do, such as improving the documentation, collecting more feedback and stabilizing the API, and so on.

Reply on news.ycombinator.com