david-j-vujic

Thank you! I will try these ones out.

I’m using Elpy because it comes with great code evaluating features (sending expressions to the REPL and such). In this config there’s also auto discovery of virtual environments. Adding to that: black, flake8 (not yet added ruff).

In the docs, you will find more about this setup with IPython integration, if the code evaluation things is something you look for too.

https://github.com/DavidVujic/my-emacs-config

Here's docs about Polylith for Python: https://davidvujic.github.io/python-polylith-docs/

I agree! In Polylith, the built artifact(s) is an implementation detail and you can decide what to build (a set of Microservices or a monolith) when you are ready for it.

Even though you have clear boundaries between microservices, the source code is likely possible to be reused by other services. You probably have common setup (like config loaders, logging and auth) and, depending on the use case, probably also parsers and data reading features that would be duplicated or put in an external libraries.

Yes, Microservices is about boundaries, data and the organizational structure. But there is code that makes these things happen and much of that can be reused.

Yes, to me it sounds like a misunderstanding. Let's imagine that you have code put in libraries, that you can use in more than one place. For any mainstream project, you would probably have these in separate git repos and published to a code repository (such as PyPI or Clojars).

With Polylith, all of that code lives in the same git repo, and you don't publish them to a repository because you have it "right there". For Python, you reference the reusable code just as any other Python namespace package. Basically the same thing for a Clojure namespace.

Everything isn't sharing everything, but several different services or apps could be using one and the same brick (as it is called in Polylith). A brick is a small isolated part of the code (usually much smaller than a library, that is an entire feature). I hope this has cleared some things up!

Thank you for the feedback, I’ll give it a try to do something like you suggest.

This is useful feedback, thank you! I will try to clear up the Python specific docs (that part of Polylith is on me).

Good point! From my experience, having different versions of third-party dependencies is a trade-off when having code in several repos. I've been in organizations where teams have needed to coordinate upgrades of things like logging & error handling tooling, to utilize the tooling across the services. If this would live in one and the same repo (and ideally in one reusable part of the code) it wouldn't be such a big thing.

Services living in a Polylith repo are deployed independently, that's a big part of the tooling support and how code is structured according to the achitecture.

The main usage is to be able share and reuse code in a very simple way. The code lives in a monorepo, and that means you won't need to extract shared code into external installable libraries. You just pick-and-choose the code needed for each microservice or app. The tooling is useful for visualization, deployment and when adding new "bricks" with templating.

I have tried hard to write docs that explain the architecture and the usage of the tool primarily in the Python space. You will find it here, and if you prefer videos there are links in there too: https://davidvujic.github.io/python-polylith-docs/

A well thought-through and enlightening article about Software Architecture - the Hexagonal (aka Ports & Adapters) and Polylith in specific. Love it!

Sounds like REPL Driven Development to me :)

There's a couple of things you can do to configure the REPL (like auto-reloading modules that has changed during the session). I submitted a link to my blog post about this subject here: https://news.ycombinator.com/item?id=37394439 (hope it is okay to cross-post like this).

There's a plugin for Poetry that will add tooling support for the Polylith Architecture (I'm the maintainer of that plugin). Here's the documentation about the Python-specific tool and you will also find information about the architecture in general: https://davidvujic.github.io/python-polylith-docs/

Another good thing with Poetry is that it enables tooling support for the Polylith Architecture (as plugins). :)

I haven’t used those tools, but I guess you would use the data types already defined in them and what is returned from the functions in there?

If you were about to pass on the result from a calculation to somewhere else, a dictionary or list would probably be a good idea. You probably wouldn’t want the entire system be aware of a Pandas specific data type.

Would you agree that it is well-known and simple?

Sounds like a good balance between the approaches!

I usually would prefer having everything behind the endpoint (such as Pydantic schemas & FastAPI) as simple dicts and lists.

I agree on that it is a good idea to use patterns that the teams have agreed on.

Would you do the same if you were about to make changes in a code base with only dictionaries and lists in it? :)

What is it then?

I was referring to the simplistic and well-known way of accessing data from a dictionary, such getting the value for a specific key or even iterating the keys & values of a dictionary. There's also quite simple to merge several dictionaries, or even pick/omit to only work with the data that is actually needed in the actual context. What's your thoughts about that?

In my opinion, the biggest advantage of Emacs is the ability run and evaluate code in a REPL Driven Development style. It is very common in languages like Clojure, less common in Python. You can do a lot in VS Code and PyCharm, but I haven't found a better combination than Emacs and the "elpy" package for this.

Reflecting on the well-known Monorepo tools out there, and it seems that most of them focus on task automation and orchestration: caching of runs and doing that fast. Those things are obviously very important, but I miss the perspectives of reusing code and also the local development experience (aside from running tasks like automated tests and such).

From my point of view, the main reason to have code in a Monorepo is to be able to easily share code between different projects and have the code & tools at your fingertips while writing new code. This is where the Polylith Architecture comes in, focusing on this particular thing. There's tooling support for Clojure and Python as of this writing.

I like the suggestions about a more descriptive name, like "multi-project", that I believe describe the thing more clearly. This would probably also remove some of the confusion betweeen the words monorepo and monolith.

Also, I don't see anything bad with an organization having more than one "monorepo" for grouping a number of related projects - an example would be different monorepos for different programming languages. This would probably mean that the organization is a "polyrepo" one (or maybe multi-project).

The Polylith Architecture support these kind of scenarios: one monorepo for all code, or a number of multi-project repos within an organization. There's also tooling support for this architecture, currently for Clojure and Python.

I'm the maintainer of the Python tooling, that is a combination of two Poetry plugins - one with the name "multi-project plugin". Here's the docs for the Python tool: https://davidvujic.github.io/python-polylith-docs/

Interesting, I’m going to give this a try!

Favorite from the Python builtins is Pathlib and the caching abilities of functools.

One of my favorites is the Rich library, making it simple to develop nice-looking CLIs.

This article isn't about Franz Kafka or his great novella The Metamorphosis, where the main character one day realizes that he has transformed into a Human-size Bug.

It is about a different kind of Kafka: Apache Kafka, with examples on how to get started producing & consuming messages with Python and Polylith.

I think I understand the reasoning behind the strict usage of pip and the Python version usage, keeping things less complicated & without much surprises. On the other hand, keeping it safe with not using the latest stable Python is sometimes painful. New features are often good things for development and code readability, such as the quite new merge dictionaries syntax and less verbose usage of the typing module.

Keeping it simple with only using pip would work for the more straight forward & simplistic kind of projects. But you would probably want something like Poetry to have a nice Developer Experience - and the abilities of Poetry plugins like Polylith to handle monorepos and sharing code.

Running a full-blown 24/7 running service is sometimes not a good fit for a feature. Serverless functions (also known as Lambdas or Cloud functions) is an alternative for those single-purpose kind of features, and Polylith is a good choice for this.