mrdoops

From my own experience in the Elixir community watching libraries develop the non-blocking preemptive actor-model concurrency is a good default for most situations. Dropping into things like Rust NIFs has been great for those hands on situations where direct memory control is needed - but more often than not its the exception not the rule.

Phoenix Liveview and the many other copied featuresets in other web frameworks are sending dom diffs over a websocket connection where a tool like morphdom will render only the necessary changes to the page.

This sort of architecture for static rendering is a big deal because it gives live page updates in a way that is SEO friendly and significantly less complex for development than SPAs are.

(Worked in the residential solar industry on and off almost 10 years)

Financing providers, sales commissions, and profit + operating margins of the typical end-to-end residential solar sales and installation company make up a large portion of the price. What you're really paying for is convenience and expertise to understand the equipment, hire/train personnel, and jump through local - often arcane municipal requirements.

However if you measure your roof and can can find a freelance designer to draft up some CAD drawings that can get approved through your local municipality, buy the equipment yourself, and hire a crew + electrician to install it direct - you'll save quite a bit - at least 5-20k depending system size. Its more leg work, but you save enough its probably worth it. The install crew, electrician and CAD designer are the only real experts you might want to hire in for. The rest is paperwork, sales, and financing.

A pretty common $/watt is about $3.5-5/watt. Expect < $2.5/watt if you're cutting out sales and financing - doing more of the process yourself.

The other metric to look at is your estimated and actual production/panel ratio. You have flat costs on equipment but depending on where that panel is on your roof (based on shading, exposure, azimuth, etc) you might get less or more for your buck. The math is costs/month based on your financing taking into account expected production (plug numbers into a tool like https://pvwatts.nrel.gov/). A good production/panel ratio can significantly affect your ability to collect savings.

I emphasize production/panel because most of these Solar Sales companies pay their sales guy on a per/watt rate where bigger is better, not necessarily efficiency and financial benefit for the customer. This can incentivize them to push for panels in places on your roof where they'll hardly produce - adding to the overall cost of the system something that is good for them and not necessarily for you.

If you have the documents and can put good numbers together you can likely get financing through your local bank directly - or imo just pay cash if you can. Avoid PPAs, stick to loans with good rates or cash.

A really common bottleneck at these solar installation companies is either 1. permitting, or later on: inspections and interconnection approval (City/Municipality + Power Co.). The shadier solar companies can technically get paid by the financing provider prior to a customer's system being fully inspected and interconnected with the grid. This doesn't always give them in incentive to bring the project to a closure as they've already gotten paid and any additional work is a truck roll, technician hours, and troubleshooting - even if it leaves the customer without a producing system and monthly payments.

Serverless systems in almost every implementation are still stateless or limited stateful-serverless (the best examples are probably Microsoft's Durable Functions or Flink's Stateful Functions - there's also solid attempts by the Akka folks).

These are largely built on top of either a stream processing (in case of Flink) or some flavor of Actor model based on event sourcing for peristence.

What I'm getting at is that you'd build a serverless system on top of Elixir/Erlang/OTP and be served well by its mature actor model implementation and excellent distribution options. See https://eigr.io/ .

The cloud provided options are interesting for some experimental / tactical use cases but I don't yet trust their abstractions to not be leaky and they typically come with vendor lock-in trade offs.

For I/O, concurrency, fault tolerance, and distribution it really doesn't get better than the BEAM, but when problems get CPU intensive you can't beat C/C++/Rust. So this idea of using pre-compiled Rust NIFs from Elixir is a huge deal because it means you can have the best of both worlds in many scenarios.

Clean modern language with top tier tooling as a coordination layer and API + bare metal speed without needing the full Rust tool chain on every dev's computer. <3 <3

Elixir. That's it. There's a reason Discord and WhatsApp run on the BEAM. It does real-time with the least pain.

Definitely a cool piece of software - there is a lot of unsurfaced potential in distributed graph databases - good luck I hope the future is bright.

In a backwards way this validates the general decentralization movement crypto is a part of even more in that a mob attack on social media can get an organization like Mozilla to limit their own capacity to accept donations.

Neat thing about Luerl is that you can manage many isolated Luerl vms from within an Erlang supervision model which is a very desirable trait for things like games with user defined scripts that may explode.

Yeah offline or complex client side state management is a good use case for Javascript. There are Hooks and Push Events with Liveview for real time integrations with Liveview in those scenarios. In my experience offline requirements are rare and often in mobile scenarios where a native or Flutter-like approach is a good option.

Complex client side state or collaborative features might use something like https://github.com/slab/delta-elixir or https://www.inkandswitch.com/local-first/ which is where I'd want JS.

React might make sense for some specific components wired up with hooks in Liveview for real-time interactions. So if you have a complex client side component that is already available in React and you don't want to recreate it - I'd just add the JS dependency to Phoenix and embed it in a view somewhere. There are definitely complex client side interactions where integration with Liveview would be preferred, however in those cases React isn't necessarily the best option - vanilla JS or D3 or something of that nature tends to be the winning ticket.

TBH I wouldn't greenfield a new web app with React if I could avoid it - either wrapping a Liveview in a React component or React in a Liveview the purpose would be to gradually transition away from React running the whole DOM/view for a large existing app.

A GraphQL server with Phoenix/Absinthe for a multi-platform app web + mobile (android + ios) is reasonable but still a bigger company/team solution. React + State management like Redux is a ridiculous amount of overhead by default and I would never implement a greenfield web app that way in 2021.

A Liveview interaction follows something like

Backend State <query/command> Liveview <websocket> Client/Browser (where only minimal diffs of changed state is sent over the wire)

and React is

Backend State <query/command> Controller <JSON/HTTP> Axios <Redux/State Management> Virtual DOM <> Browser/Client (where whole json payloads are sent and encoded/decoded over the wire)

Where new state means re-fetching all the state each loop instead of just what changed. The amount of data sent over the wire by default is massive with this React + HTTP/JSON model.

The encoding/decoding of JSON and having to send full payloads then having state management occur in multiple places and languages - it ends up being a lot more to know and think through. The lines of code to write and amount over the wire with the classic JSON over REST + React/Redux makes it almost 100% required to have separate front/backend developers which then means coordination costs for every new feature.

So TLDR; React could make sense for complex components you don't want to reinvent the wheel for or when those interactions are complex and client side. Existing web apps with a lot of React code that is too much to port over all at once can be done piece by piece (where I'd recommend heading towards handling navigation in Phoenix/Liveview and components in React).

Elixir is modern tooling on top of the BEAM / Erlang Virtual Machine. This is tech that has been running Telecom systems, companies like Whatsapp, Discord, Klarna, and software like RabbitMQ, etc for decades. The core libraries are very active and well maintained.

I tend to prioritize "capabilities to build compelling systems" which, to me, means getting to production quickly so user feedback / product-market-fit tests can occur, but then also scaling a team to maintain the application over time because once you've found a good fit its time to grow.

To me stateless CRUD isn't compelling. Its too easy - if your app just provides a GUI in front of a database for Create/Read/Update/Delete operations your app is a database not an app and if all it takes for someone to clone your app is 5 minutes of CRUD generators, then that's not exactly a defensible position to build a business on.

Phoenix like other web frameworks provides generators and quick paths for your usual Web + Database applications. Run some commands and you've got a web app with database persistence. Everyone can do this nowadays (Django, Rails, etc) What Elixir and Phoenix does better than anyone else is distribution, concurrency, fault-tolerance, and real time streaming. These capabilities are possible because of deep details on the virtual machine level and become trouble for cooperatively scheduled runtimes elsewhere (e.g. Python, JS, Java, Ruby, etc).

If you're building something with: websockets, collaboration, chat, i.e. anything real time between many users - save yourself the trouble and look into an Elixir/Erlang solution like Phoenix/Liveview/Channels/Presence.

On the human side Elixir has top tier documentation (example: https://hexdocs.pm/phoenix/channels.html#content) - we've found it doesn't take long for developers with at least a few years under their belt to pick up the language/framework and be productive. You don't have to know everything - just build some familiarity of where to look and you'll figure it out. Its very important that this human side be scalable as well, and that means developer experience, documentation, on-boarding, etc.

I've been using Elixir/Phoenix every day since 2017 to build apps, glue systems together, rebuild Ruby/Rails/PHP/Python apps that don't scale anymore and in general just ship features on time consistently.

The thing that I'd toss in as an "it depends" on the technical scaling side is Postgres - its a boring single server relational database that just works but is still a single server architecture made for spinning disk persistence. SSD/NVME/Persistent Memory and more are new foundations for persistence which means new databases on the new foundations may warrant attention soon. 99% of the time just use Postgres/MySQL though. If you're not sure just use Postgres. For many apps in finance or with stringent auditability that are log-centric using a time series and/or event sourcing database makes sense for the projection / read model flexibility. That said you can use Postgres for that so there's no good excuses for not using it.

I'd say the biggest disadvantage/con to using Elixir/Erlang is that its more of an expert's tool-set. These are sharp tools made for engineers that make sense to engineers with experience to know they want fault tolerance and concurrency. What this means is that a newbie developer might not know what they don't know they want to know when building an app the first time. This caveat is probably true with any tech stack, but there will be a lot to learn about the backend because there's more possibilities on this backend stack. These backends-as-a-service (Firebase, Hasura, etc) will get things rolling quickly but in my experience all the real juicy core problems of your business will end up in backend code and when that happens you'd better hope those quick solutions don't get in the way.

Phoenix + Liveview + TailwindCSS + Postgres

Unless a static site or buy option fits right, otherwise it doesn't get any better than a Liveview based app for continued productivity.

Yep and this approach also allows languages like Julia and Elixir to compile their expressions into valid compute graphs that target JAX/XLA. That polyglot capability opens up cutting edge machine learning into quite a bit more ecosystems with another level of capabilities in distribution and fault tolerance as is the case with Elixir + Nx.

For getting the business off the ground - there's often a build vs buy question.

My rule of thumb is buy for generic build for core.

For SaaS/web app aspects? For me, probably a flavor of the PETAL stack i.e. Postgres/MySQL + Phoenix + Liveview + Tailwind. My core is usually some kind of state machine modeled in boring structs and functions. No frameworks - no libraries if I can help it. Pure as possible when possible for core components.

Buy over build (No code / Low code / SaaS / Spreadsheets / CRMS / etc) if its generic and not a core concern. Lead management, a lot of Sales, marketing and case/customer service ticketing concerns in a lot of businesses are often good candidates for buying over building.

Never buy if the software is a core business capability - write that code yourself if you can. You can avoid a lot of maintenance risk over time this way.

The worst case scenario is you ship quickly and don't find a product market fit, so ship quickly for that feedback.

The next worst case scenario is you found a fit but your core business capability is tied to a third party tool you don't control and it starts getting in the way.

No API? Don't buy. No real-time API in 2021? Probably don't buy. There's good low-code tools but there are also predatory SaaS tools out there that can enact suffering and delays to your business, so weigh that decision carefully.

They need stable, actionable futures. Homes they can afford to own, communities they can raise kids in, an environment that won't go up in flames; they need systems that have their back.

Build something better.

More demand -> increased prices for developer time. Greater costs / impact / scale of software development -> greater risk to the collective stakeholders. Greater risk -> more potential for reactionary measures to implement more stringent hiring / control mechanisms.

That's how you end up with many many interviews, lengthy code exercises, and huge opportunity costs for everyone involved.

Consequently the same pressures would apply to software project management procedures towards growing complexity, and less agency/autonomy.

Why does it seem like 'AGILE' often transforms into less of an idea of short feedback loops, context and communication and more into a bureaucratic control system? We implement control systems to reduce state space of unknowns / chaos and trade-off variance in favor of consistency. So if software in organizations is under high demand, increased cost, and greater risk it would also suggest likelihood of similar reactionary measures to implement control systems measures on software development as a whole.

Consequently the ever present trade-offs of exploration vs exploitation or global vs local maximum would also apply meaning the reactionary measure of increasing control and consistency has side effects counter to its very purpose: improving software capabilities. Instead of increasing stringency / consistency through process/procedure also increasing cost of interaction one would instead want to reduce cost of interaction, but also reduce costs of downsides upon hiring (e.g. contract to hire + tasks to prove but where acceptability of failure is present). This has the added benefit of not accidentally removing "exploration" from the equation allowing higher risk higher reward employees into the organization which reduces risk of overfitting and innovator dilemma problems.

Want to build a real time chat app? Best build it on the BEAM.

Agreed. I would bet on a language like Elixir or F# being simpler to learn and grow for a complex system than a class-oriented-imperative-oop (Java, Ruby, Python, etc) language any day.

Are there plans to implement a distributed stream messaging system (e.g. Kafka, RMQ streams) embeddable in a supervision tree like Phoenix PubSub?

Way easier said than done I'm sure, but I know personally there are a lot of use cases for the distributed stream guarantees.

They're also separating out Phoenix.View so there will be less dependencies on using Phoenix view rendered templates for things like email or PDF generation outside of a web layer.

The whole approach of regulating on the level of "please don't exploit vulnerable systems" seems reactive to me. If the cats out of the bag on a vulnerability and it's just data to copy and proliferate - not much a government can do other than threaten with repercussions which only applies if you get caught.

The only tractable way to deal with cyber security is to implement systems that are secure by default. That means working on hard problems in cryptography, hardware, and operating systems.

I expect streaming will be adopted like anything - first in smaller more mobile teams working in greenfield contexts, then later in large organizations when the idea is less novel and lower risk/cost to apply at scale.

Regardless I feel streaming has been around long enough now it should be urgently considered anywhere where the words "data pipeline" and "scale" are thrown around together frequently.

I think that's a bit of a reduction/straw man to say they need a lot of architecture/central-planning. An individual developer can do it so long as they have access to domain experts to ask the right questions. Lack of proper planning will result in any architecture failing miserably - best not to code until we know what to code.

The implementation overhead as far as code-to-write for streaming architectures is comparable to CRUD. But there is less knowledge dispersed about practices on how to do it, so there is the cost of learning. It is more cutting edge after all.

From a technical perspective there are plenty of ways to serialize / order some stream of events in a reasonable way. Whether that's implementing your event store on top of a transactionally secure database (e.g. Postgres, not Mongo) or using a higher throughput, less persistently secure solution like Kafka. There's also ways to deal with eventual consistency and distributed transactions (SAGAs) depending on the need.

The hard part is that ES/Streaming systems work best / almost necessitate a clean and clear domain model. A clean and clear domain model requires a lot of discussion and consensus with domain experts and product owners. Buy in to have the kind of discussions needed is the source of the issues I've experienced with these kind of systems. CRUD can paint over a lot of cloudy abstract concepts for better or for worse. These kind of discussions are energy intensive and mentally painful to cast light on the cloudy thoughts.

There's not great streaming/es support on a language level outside of the robust actor model systems (e.g. Erlang/Elixir). There are systems like Akka that simulate that to some extent on runtimes like JVM, but a cooperative scheduler and an actor model don't mix great. For non-actor model aspects I've been seeing more service level dataflow systems like KSQL / MaterializeDB gain traction, but are nevertheless a solution for read-models not application logic.

A lot of comments not sure about what Graph DBs are good for:

* Flexible knowledge association i.e. Knowledge Graphing

* Modeling and querying associations / models with many-steps-removed requirements

* Expert Systems / Inference Engines

* Lazy traversal for complex job scheduling

Graph DBs are not good at being a general purpose 95% of use cases database. Just use Postgres/MySQL if you're not sure. We use Neptune (AWS managed GraphDB) to model cybersecurity dependencies between many companies and report on supply chain vulnerabilities many steps removed. Those kinds of queries are non-trivial and expensive on anything but a Graph Database.

As GraphDBs meet niche query requirements you usually have other databases involved in the full application. If you want to tractably manage many databases in a system you ideally want to be in streaming / event sourced semantics. If you're already in an imperative crud-around-data / batch pipeline you'll find greater maintenance costs in adopting a GraphDB or any additional DB for that matter.

No-code tooling or not someone still has to design the workflow. Designing a workflow requires asking good questions, which requires putting in the time with domain experts who aren't necessarily an expert in implementing a workflow with software.

You'll just get different flavors of specialists this time with newer no-code tools. IT will still probably be doing this work. Writing custom code / using generators might still be more optimal to avoid vendor lock-in that is often the revenue strategy of low-code SaaS.

Already planning on running this in an internal cluster for training & documentation scenarios. Hexdocs are great, but having a live executable tutorial connected to a staging database sounds really valuable for getting new team members up to speed.

OTP is definitely more of an expert's toolset than a framework for throwing together features quickly. This is a good thing because it's not trying to be more of an abstraction than necessary. With OTP you can make fault-tolerant, concurrent, self-healing applications that handle stateful operations with ease, but you'll still need a fair bit of need-to-know about the callbacks, how to manage process lifecycles, etc.

What a lot of people seem to want is a batteries included framework that does the OTP for you, but doing at a high level of abstraction is difficult. So more commonly you'll see the OTP managed by libraries for more concrete use cases (web servers, database connection pooling, back pressured job processing, etc). Trying to wrap or rebuild OTP from the primitives available in the BEAM is possible, but definitely not an easy task.

Could we potentially have an OTP-like library that cuts down on the boilerplate significantly? Yes. But how much effort would that take and is it much of a win over OTP as-is? Not sure the trade-off is entirely there yet - OTP really isn't that bad. You're still writing less boilerplate than Java in general.

I think we're much better off writing good library-level abstractions over OTP that can be embedded in a supervision tree or used in more specific use cases. If we were to actually attempt some kind of higher level abstraction in the scope of OTP - maybe we can do so with a very different approach like decorated dataflow graphs and runtime property based testing.