You can serve static data over HTTP
ignore.pl
ignore.pl
Indeed. I've noticed an increasing number of people who call themselves "developers" and appear to create software, but all they can do is follow step-by-step tutorials to glue together some massively bloated thing that they have absolutely no understanding of, much less the skills to debug it when something doesn't work perfectly as expected.
That won't really happen, right? No Computer Science (CS) professor would designate special cloud Interactive Development Environments (IDEs) to students.
On the other hand, it would really resolve a lot of "doesn't work on my machine" issues.
Setup some obscure software, with poor documentation, build something nice on top of it, then rewrite it because some fundamental assumption is broken, sounds like this kind of first year of cs course we should have.
The idea being weeding out those not as passionate about computers.
GitHub has said one of the largest growing areas for Codespaces (cloud IDE) usage is Education.
It's already happening. It does resolve a lot of "doesn't work on my machine" issues; it does leave some questions about how much students see of the "ground" below the Clouds.
Welcome to the world of npm and supply-chain hell
Here's what I mean. There's a difference between:
* Using an IDE knowing what you want to do but you just like the convenience. You know what the IDE is doing for you, but you want your tool to do it for you quickly and get out of the way. If the IDE does something you don't know, you try to learn what this was.
* Using an IDE because you literally can't work without one. Not knowing your programming language and depending 100% on autocomplete to do most stuff. Not even knowing how to write the files that the IDE automatically updates when doing certain things.
It's a subtle difference, but the former is more likely to leave better comments in a web-based code review ("pull request") than the latter.
And if a project consists only of the latter type, it's unlikely that there will be comments like:
* "Use this other class with the same name but different package because this one is buffered" (because depending on autocomplete without knowing the language prevented them from even caring about which import to use, or the difference between those, as long as it works).
* "Would this change affect this other file that's not in the PR but calls this function a thousand times per request?" (because not knowing their own projects general structure due to depending on their IDE for navigation, so they're more likely to NOT notice a file that's not mentioned in the PR).
So if a project has only people that depend on an IDE (instead of using an IDE for convenience), it's easy to end up with stuff that technically works, but copies 40MB of strings around multiple times, and thrashes your performance due to reading one character at a time without buffering.
Granted, this technically is not related to IDEs, but in my experience (emphasis on "my"), most of the people I've worked with that use an IDE, are of the "can't program without it" type, and not the "I use it because it's convenient" type.
Not all of them. But most. Even if there's a learning curve at play here, I'd say there's usually overlap between "use IDE because convenient" and "actually bothering to put an effort into learning".
I understand this is an inevitable consequence of lowering the barrier of entry and making programming more accessible, and it doesn't matter if we're talking about npm, IDEs, programming languages, or anything.
Pretty much this. Anybody new to the field will be using an IDE, it's extremely unlikely that someone new will be using vim/emacs. So IDE users will be a mix of newbies and experienced people, but since the field grew so much in recently, the number of newbies using IDEs will outnumber the experienced people using them. In contrast, people using vim/emacs will consist mostly of people with a lot of experience at this point.
So the average IDE user will be less experienced than a non-IDE user, but I don't think it has anything to do with the fact that they are using an IDE. I wouldn't attempt to pre-judge anyone's ability based on if they use an IDE or not.
Neither would I, but I will certainly judge them as lacking if they can't write a nontrivial program without using code completion or looking things up on SO/Github/etc.
Does this arbitrary bar extend to language and library documentation as well?
I certainly have way too much in my head to remember every nuance of several languages. As such, I lean heavily on multiple sources of external documentation to help me recall the specifics. That being said, this would in no way replace actual software development expertise, so I agree with your clarified stance.
I find using a good IDE with code completion and navigation is essential to efficient use of my time. I _also_ use vi constantly and perform most every other operation from the command line. I know I can get the same from properly setting up vi with the right plugins, but I find the ergonomics less appealing for my habits.
Sure, maybe there's some design principles to share that aren't obvious. But it's hard to share and teach those before someone has experience with at least putting stuff together. Also, it's not like there are principles that are universally accepted or universally applied by trained software engineers.
The way to bring a cargo cult developer into the light typically involves debunking or contextualizing a whole bunch of weird ideas.
Knowing how to apply design principles and rules despite not fully understanding them is a different level, in my mind.
Soon we may have no-code solutions getting good enough to warrant broader adoption. No-code solutions are as black box as you can get, and it's also as anti-learning as you can get in terms of the underlying technology and its theoretical principles. But this is also an example of more people than ever making things.
A tool which only allows perfect solutions by perfect users is one that rejects a lot of work and prevents a huge number of projects from ever happening at all. Useful for safety-critical work, but not everything is safety-critical.
As in be a terrorist and produce bombs? These are all ticking time bombs.
Relevant: https://xkcd.com/1988/
Lego's dream has come true.
caddy file-server
Also:
caddy respond
caddy respond --listen :2000-2004 "I'm server {{.N}} on port {{.Port}}"
cat maintenance.html | caddy respond \
--listen :80 \
--status 503 \
--header "Content-Type: text/html"
Caddy excels at making simple [1] use-cases trivial - either with no config, or 3-5 lines of config.
It might seem jarring comming from Apache or nginx - but i think a lot of it comes from the developers having destilled a lot of common use-cases and carefully chosen reasonable defaults.
[1] Simple to articulate; not necessarily simple to execute - eg letsencrypt works out of the box. Caddy will give you https if you ask (or rather unless you go out of your way to avoid it).
It can be a great stop-gap in running an ssl proxy in front of a legacy application server stuck on some horrible old distro, for example - since the binary just needs a Linux kernel (and can ignore old ssl libraries).
To be fair to NGINX, there is a lot of boilerplate config, but if you have a good set of default files, the server blocks can be small. The same could be said for Apache, of course. But yeah, Caddy is great in how little config it takes, no boilerplate, to do what the incumbents do.
At my work I struggle with the opposite: all problems are being squeezed into "let's put it into static JSON on the CDN" - which ends up with a complex custom JSON based language (schema) to support sharing information between apps, subsetting information (ie. search) etc - ie. implementing an ad-hoc one-file database. Ahh... and don't forget about complex CI/CD pipelines and Git setup that allow business users to manipulate data in these files.
I found "just use Postgres for everything" a much saner default in the end.
So it’s simple or complex because I got confused ;)
KISS is ALWAYS good. But simplicity doesn’t mean easy or naïve solutions. Naïve solutions get very complex since the initial input is low and they don’t cover edge cases by design making it tangled mess.
Simple solutions are often very difficult to create, require a lot of input to be simple, but they cover a lot of cases behind the curtains and are easy to describe.
Your “use Postgres” strategy is more KISS than doing complex mocking. Sure, initial effort is higher, but using well known, powerful yet ergonomic singular entity is as simple as one can get on some levels. It’s equivalent of “just use Excel for it”.
I agree with you, but I think tagline is misplaced.
So yes, Kiss is good until it isn't... because it wasn't kiss anymore. Moving to the postgres solution sounds like it's the new Kiss.
I don't know how this will ever not be a complete maintenance hell. It is even far worse than the monolithic "I push every bit that needs to be pushed for the whole enterprise - Sync Tool".
Best of both worlds.
People should not cache something that changes very often.
The fuller version of KISS is "keep it as simple as possible, but no simpler".
We also do something similar on Datatig, which is a tool I'm writing for the general situation of crowdsourcing data in Git repos. It makes it easier for people to submit data and then creates usable data and handy websites. The static website output has an API which is just static JSON files. https://pypi.org/project/DataTig/ or https://datatig.readthedocs.io/en/latest/
Makes it super simple to implement mocks of your api design before building the api.
…I guess that was kind of the point of REST in the first place but it’s easy to forget.
REST has a wonderful simplicity to it.
I’m yet to hear about an equivalent caching solution for GrqphQL APIs for example.
It’s been running unattended for a decade….
> We’re going to disrupt Internet hosting and serve static files in a novel way through a cloud based SaaS.
Yes. Just like you can put letters together and form words. It's kind of what it was designed to do.
Why on earth the markdown was not pushed through a renderer so that then the server could serve static HTML is probably a question that only occurs to old timers like us.
A generation that grew up with Wordpress et al. will have dificulties getting that really all there is to a website is some html. I switched from a Spring Boot web app that used Thymeleaf for the templating engine, with a MySQL db on a Linux vps and Heroku (I did it to learn/practice) to a Hugo blog that I publish locally then scp to a VPS with Caddy setup to serve the static content. Nothing to hack/maintain/worry about.
As an extra point I would even complete the suggestion with adding direct HTTP not S, support. Your page will load even faster and you're transmitting some static content anyway.
Best as I can tell the part that might be non-compliant here is that new followers should be added to the follower collection.
You need something to deal with follows (cryptographically verify that the follow attempts actually originate from the servers they claim to) and submitting posts to your followers (push based, not pull based in practice) but all the metadata and user lookup stuff can be done with JSON files in the right place.
I once spent an entire month building a proper API server out of this. Then one day while tweaking some DTOs for some POST endpoints, I realised what I was doing and switched to Postgres.
That also tends to provide results faster than the database layer, so better latency and less database load.
It's not as fast as serving static content via nginx or a CDN though. :)
Anyway, as everything in software development: it all depends - serving a simple JSON over an existing "full blown web application back-end" is usually more convenient and less confusing for developers than configuring hosting for static content. Although, if you JUST want to serve a JSON then, yeah, you can add it as a static resource.
But only for prototyping and testing the auth and database tools.
Haven’t really made a decision on it vs rolling my own in Rails.
This is essentially MITM-as-a-service.
For the rest, even for a prototype, it seems a lot of trouble to insert, update and delete data.
But on a(n only slightly) more serious note, maybe the data is generated outside of the web server. Different projects have different needs, but there are plenty of them that just need to serve some mostly static data.
I first came across this when creating a demo for a smartpen that was supposed to have live interactivity features, which I was going to implement by putting a little HTTP server on the pen.
When it came to simulating my solution, I took away features until I came up with the "serving static files over HTTP" trick the post describes.
Then I realised that I could do the whole thing by just putting the demo files in a directory and changing the base URL in the client from https://... to file:///...
Another time we built a backend system for feed processing and delayed putting in a database until we discovered we didn't actually need it.[1]. The standard operating procedure at the time was to create an Oracle DB for everything, including a system that needed a total of 6 data items. Wow.
My boss at Wunderlist used to brag that the database we used for the iOS/macOS clients was just a bunch of JSON files in the filesystem. Worked wonders and was really, really fast.
These and many other experiences led me to come up with Storage Combinators[2]. Once you try them, you'll wonder why things always have to be so complicated.
And yes, I agree with the idea that "gluing complex libraries together that we don't fully understand" is a problem. And also with the notion that it's an achievement that we can and should be proud of, at least partially, because that used to be something we couldn't do at all[3]. Nowadays we can at least do it badly[4].
My take on this is that we don't have the right kinds of glue, particularly procedural/functional abstraction is often no the right glue, and OOP notwithstanding that's really all we got. [5][6]
[1] https://link.springer.com/chapter/10.1007/978-1-4614-9299-3_...
[2] https://2019.splashcon.org/details/splash-2019-Onward-papers...
[3] https://blog.metaobject.com/2021/07/glue-code-is-success-con...
[4] https://blog.metaobject.com/2021/06/glue-dark-matter-of-soft...
[5] https://blog.metaobject.com/2019/02/why-architecture-oriente...
No, it doesn't. Maybe for your personal homepage with three hits per month, but even on a low traffic server, static files gave me headaches.
1) When two visitors write to the file at the same time, the data of one of the visitors will be gone. This can be solved by using lock files. This is a topic on its own (what happens when two processes both create a lockfile at the same time?). Also, when there are a lot of writes happening, multiple processes will wait for the lockfiles, creating load on the server and wait times for the visitor.
2) When a file is read during while another process is doing a write, the file is only read partially. There are workarounds for this as well, you can first write to another file and when done, move this other file to the correct file.
As the number of visitor increases or the file size increases, problems will get harder.
Nowadays it's not hard to add a database. Whether it's on your server or in the cloud. It's performant, it works and it will grow with increasing demand.
It's probably easier to set up than a file based solution.
Sometimes what seems to be simple isn't really.