The good parts of AWS: a visual summary
hassenchaieb.com
hassenchaieb.com
I would absolutely recommend S3 for static web hosting. Just add CloudFront on top if you need HTTPS, it takes like 2 clicks.
If anyone interested, I answered on StackOverflow how to deploy a front end app (React, Vue, etc) to S3: https://stackoverflow.com/q/41250087/652669
I'm a bit stumped on why they say to avoid using local indices in DynamoDB, though.
https://docs.amazonaws.cn/en_us/amazondynamodb/latest/develo...
Here is a link to the AWS docs referencing the limitation of local secondary indices. Any table with a local secondary index is limited to 10GB per partition key.
The authors may have misinterpreted this limitation.
>Once you create a local index on a table, the property that allows a table to keep growing indefinitely goes away.
It limits the value for that partition key but does not limit the overall size of the table...
I used this stack (S3, CF, R53, & ACM for SSL cert) to deploy a static site built using Hugo, which can deploy directly to an S3 bucket.
If you're crafting the site yourself without a builder, you can just use the AWS CLI to `cp` the files to the S3 bucket. You may need to invalidate the CF distribution if you want everything updated quickly worldwide.
The recommendation against using S3 for static hosting is that there are much better alternatives outside of AWS. The dev experience you get with Netlify and Zeit will almost certainly never be possible with AWS.
"""
#!/usr/bin/env bash
rm -rf public
hugo --verbose
aws s3 sync ./public s3://$BUCKET_NAME --profile s3_personal
"""
You can set custom TTL for CloudFront to 0 for instant refresh in production. Dev server is `hugo server`, `hugo` is a single Go binary you can Dockerize with --net=host and --volume=$(pwd):/app to use your own dev environment. If you don't maintain too many large static assets, your cost is like 5 cents a month, and I think that's a competitive price, not AWS being generous. $12 / year for a custom domain via ICANN, and Route 53 manages A/AAAA/CNAME/MX records for free.
Again, not to refute your claim. I'm a single author, who publishes as a hobby, with a single dev and a single prod environment, so if things break down (it hasn't for years), it's easy to fix. It's much different than content marketing and enterprise use cases. Even so, I would rather start here and add a static CMS like Forestry before considering a custom solution like Netlify (which is probably built on top of AWS anyways).
Looks like you can add extra developers on the lower tiers for $15/user/month, but they're still charging way more per GB than AWS does.
I get that you should always charge more if possible, but I'm much more concerned with surviving during hard times, and there's just less to ablate away if you stick more or less to first principles.
If your time frame for hosting is 10+ years, I’d personally recommend to put your eggs in s3, (or the “s3 interface”, to be specific; whether it’s with AWS, DO, etc).
So, is Amplify just a simplified interface to S3+CF?
Includes CD and Amplify itself (like Cloudformation) is free, you only pay for the resources you make.
AWS Amplify has been out for a couple years now, and it is also able to automate much more than just static websites.
So taking a look at the 2 options you mention, I am not seeing anything that those do that Amplify does not (but the opposite is not true, Amplify does a lot of things for you out of the box)
For static hosting, AWS Amplify is already a very similar experience to Netlify. Each is slightly better than the other in some ways, but overall they are really very similar.
I don't understand the eagerness to trade-off $/user instead of $/bandwidth (which is far, far cheaper in nearly every case) just to save a day or two of learning how to use a tool.
With the recent GA of AWS CDK [2] I would say that it is getting more and more possible to define your own development workflow that is tailored to your own static website hosting needs and better than just raw S3. AWS CDK is still difficult to use with some rough edges, and I know that setting up e.g. a CodePipeline and CI/CD is difficult.
Here is the CDK example for hosting a S3 origin + CloudFront CDN + custom TLS certificate + CloudFront invalidation on deployments static website [3]. Here is my personal version of it combining Hugo that took me a day to set up [4]. Still complicated, lot of rough edges, but offers a pleasant dev experience [5], I haven't yet built up a CI/CD pipeline.
Disclaimer: I work for AWS but my views are my own.
[1] https://aws.amazon.com/amplify/console/
[2] https://aws.amazon.com/cdk/
[3] https://github.com/aws-samples/aws-cdk-examples/blob/master/...
[4] https://github.com/asimihsan/asim_ihsan_io/blob/master/cdk/s...
[5] https://github.com/asimihsan/asim_ihsan_io/blob/master/READM...
That's exactly what Amplify Console is, and it's really good.
If you truly have no interest in them other than as a user, then it seems as if you're not aware of alternatives such as AWS Amplify. You haven't responded to any of the comments pointing out the issues with your claims.
Whatever book you're a co-author of doesn't sound as though it's worth the price.
As far as I can tell, AWS exists to optimize accounting and inter-departmental friction, and not to solve any technical problem.
For convenience and other reasons, I use S3 to publish some videos for a small group (~30) I teach. Wondering if serving that thru CloudFront would be cheaper.
This is especially important to me now as I'm doing this while everyone here is under voluntary self-isolation so I'm relying on video more.
Not that S3 was ever that expensive for me but if CloudFront is cheaper...
S3 transfer to CF is free I think so might be worth my giving it a try.
Otherwise, it might not be worth the cost of the distribution!
I'm paying for either S3 or CloudFront for distribution regardless, and either way their transfer out to internet pricing looks similar:
https://aws.amazon.com/cloudfront/pricing/
https://aws.amazon.com/s3/pricing/
Transferring from S3 to CloudFront looks free, so I might give it a try. Already have S3, just have to set up CLoudFront.
I hope I didn't miss something on pricing... really can't afford bill shock!
> Choose between [DynamoDB] on-demand option (no capacity management) or provisioned option (cheaper).
It's very, very easy for provisioned capacity to be more expensive than on-demand. You need to overprovision because, if you exceed the provisioned throughput, you will get throttled and your application will suffer. Scaling up and down is a slow, opaque process, and you're only allowed to do it something like 5 times per day. If your workload has any idle periods, you're just burning money.
> Avoid using S3 for static web hosting (No HTTPS)
I won't argue that S3 is ideal for static web hosting, but "avoid" is pretty strong and IMO not warranted.
> Do not use AWS Lambda as a general EC2 host
What does this mean? As general compute? This is too simplistic a judgment. Lambda is a versatile service and can be useful in many situations.
> Kinesis: unlimited consumers
No, Kinesis is an extremely limited service. 5 reads per second per shard. In my experience, if you put even two consumers on a stream, you will start to see throttling. The official solution is to fan out to multiple streams. Hacky and super expensive.
> Kinesis 30x cheaper than SQS
This is hilarious. Maybe it's true for some workloads. In my experience, Kinesis is incredibly expensive and SQS is not.
Ya, "avoid" is a strong word. I use S3 for tons of static web hosting, because it is exceptionally cheap and I don't have to worry about patching CentOS/Apache/MySQL/PHP and all the other binaries/daemons that I used to. I know how to, I just would rather not when S3 is a couple pence per GB.
You can stick Cloudfront in front of it and get HTTPS very easily (with the added benefit of being on edge servers).
Personally I have found S3+Cloudfront to be perfect for static content for that reason.
It seems like they're saying "Don't use this for generic services, only use it as an integration with other AWS services" to which I would say, "What?"
If I recall, https://acloud.guru for example is hosted entirely using Lambda, API GW, S3 and some other AWS services.
https://aws.amazon.com/premiumsupport/knowledge-center/cloud...
https://aws.amazon.com/blogs/aws/amplify-console-hosting-for...
Not anymore. Enhanced fanout allows up to 20 consumers, doesn't count against read limits and has lower latency than polling.
> It's very, very easy for provisioned capacity to be more expensive than on-demand.
Reserved capacity will beat on-demand pretty handily, even with massive overprovisioning. Assuming a 50% target and a 16 hour daily duty cycle, reserved provisioned is 20% the cost of running on-demand.
I don't have the math on hand at the moment.. but switching to on-demand for us resulted in HUGE savings. We weren't overprovisioned either, but we were kind of spiky (in fact we were underprovisioned for a lot of the day).
Regarding Dynamo: I'll echo plexicle's experience that switching to on-demand was an immense cost savings (these were tables that were not being used often, many of them in dev environments).
- S3 for static assests
- SQS V Kinesis
- Lambda ('Small code that doesn't change....what).
- ELB completely overlooked, if you need session persistence then you need an ELB so for more traditional software you've completely omitted that solution because it's "Legacy".
Missing
- No discussion on when to pick Lambda, Containers or EC2
- No Cloudfront addresses S3 and any API GW or ALB fronted Lambda.
- Where is the use of SNS?
- What if I want to Fan Out on a message/event? complete missed the why SQS V SNS V Kinesis and when to use which.
- No RDS....so if i've got relational data are you advising i should put it in DynamoDB?
- No container context in any way.
Really poor quality of content and advisories needs an urgent update.
Right, the SQS part is very misleading: "SQS has 1 consumer" is technically true (although it's actually one group of competing consumers), but in practice, you get fan-out to zero or more consumers easily, by wiring SNS and SQS together (one SNS topic, multiple attached SQS queues).
> No RDS...
Without also mentioning RDS with DynamoDB, you can't make meaningful choices between them.
It also omits the one big benefit of DynamoDB: single-digit number of milliseconds to read a record by primary key, regardless of scale, due to sharding on that key.
Additionally the legacy ELB is great for when you want to do you own TLS/SSL termination with Lets Encrypt. (Especially with kubernetes, ingress, and cert-manager.) Also with the legacy feature, you can enable ProxyProtcol which passes on the public IP info after the NAT.
Kinesis has strict ordering but SQS does not?
No.
Kinesis has strict ordering PER SHARD. Which dilutes your throughput, unlike SQS. And it has limitations on throughput in terms of total event size, which are different from SQS's 300 TPS for FIFO.
It's apples and oranges. More significant, this comparison really makes it seem like you should pick Kinesis for a simple starting project rather than SQS, which is the exact opposite of the truth. SQS is the simpler choice for 90% of the cases, and Kinesis is an advanced powerful tool for the other 10%
Kinesis is for streaming data, and SQS is for job queues.
https://aws.amazon.com/blogs/aws/new-tls-termination-for-net...
As someone who has done a lot of work with AWS, I have at times in the past picked the wrong one of Kinesis vs SQS for different workloads - largely due to my inexperience with Kafka and similar at the time, and misunderstanding the point of some of the "benefits" that I saw for using Kinesis vs SQS. Most of those times, I would have been better suited to a combo SQS/SNS for fanning out (something a helpful HN reader actually suggested to me!).
Anyway, EventBridge steps in perfectly for most of those use cases, my new projects are using it, and I am considering going back to some old services and reworking them to use EventBridge instead. The rules/filtering parts in particular are going to let me decom several lambdas and save money. Definitely worth people investigating.
The first sentence re: Dynamo discusses the inability to do filtering or sorting which is flat out wrong:
“DynamoDB requires data operations (aggregations, filtering, sorting ..) to be done by your application. → All data needs to be sent over the network.”
Enable the dropping of invalid headers, enable HTTP/2, let the ALB terminate the TLS, and you'll see benefits even if your full backend isn't HTTP/2 enabled and you'll have eliminated a whole range of other headaches you no longer need to manage. It's one of the most reliable services AWS has.
that's ... only true if you're exclusively using strongly consistent reads, but if you're exclusively using strongly consistent reads, why are you using DynamoDB to begin with?
Route53 is probably the biggest value-add on top of raw compute & storage that exists in Amazon's entire cloud product portfolio.
Who else remembers requiring 3 different vendors to run 1 https website?
Highly recommended viewing: https://www.youtube.com/watch?v=HaEPXoXVf2k this talk explains how relational data can be efficiently modeled for key-value stores.
- SQS pricing is $0.0000004 per request [1] with 1 Million monthly messages free.
- SQS + SNS is far more flexible and reliable than Kinesis, plus you're paying for and managing shards by the hour. Its great for streaming analytics data processing and distribution (within the Kinesis suite there is stream processing ala Spark and streaming ingest to Elastic, S3, and Redshift). If you're not doing data engineering on a streaming set of events that can't fit in a spreadsheet, its just not the right fit.
- Lambda is not expensive to use with DynamoDB at all, the pipes are so fast that that 50-100ms is typical for a call and response. Lambda does get gnarly with SQL databases that require connection management and pooling, but they have improved that situation on multiple fronts in the past 6-9 months.
- Also, Lambda ought to be considered the first choice for runtimes for greenfield apps, not just for AWS plumbing. This is strictly opinion and definitely depends on your team and existing codebase, but it makes sense from a cost and scaling standpoint for all kinds of projects.
- NLB should be the first choice only for specialized use cases like UDP services or those that require extraordinary throughput. ALB gives you an incredible feature set with things like HTTPS, Auth, access logging, routing requests to different machines or Lambdas based on url path, canary deployments, and more. Use as much or little as you like. The cost difference between ALB and NLB is negligible for most use cases.
If you're new to AWS and cloud stuff: welcome, the water's fine. Sure its hard at first to grok all the breadth and depth of the offerings, but you get to build with the same tools the largest services on the web depend on. Start small and pick a service, learn it from the console, then cli. Hit up the docs a lot and ask questions, there are lots of people that can help. Do NOT get lead astray by this article or opinion pieces on Medium.
Something like 75% of AWS services are geared at serving enterprises with on-premise data centers that want to move some or all of their workloads to the cloud. Stick with the basics like S3, Lambda, SNS, SQS, and whatever servers you need. Pick a managed service vs servers when you can unless you have solid sysadmin resources and a good reason. Avoid hourly billed services whenever you can until you can get cost management figured out. VPC is daunting unless you're good with networking, but you can avoid it entirely by using the primitives mentioned above. IAM takes some real grokking, but there is almost nothing you can't do with it to scope permissions both internally and externally.
AWS YouTube Channel is full of quality material for all levels. Try to start forming your opinions with those and your own experience instead of unsubstantial articles or random HN comments :)