necro

Time is local Timestamp is a counter

A unix timestamp does not have different timezones. It is a counter. No matter where u are in the world a timestamp call should give you the same numeric value at the same instant. It is not time zone adjusted. Store that number, unadjusted as the source of truth. You can get to any local time after that.

I think the thinking is that if you had kids you created future cows for the tax plantation so you contributed more to the country.

Internet related would be, back in the early 2000s when I was building out infrastructure of pinkbike/trailforks I was optimizing things and discovered that this initcwnd parameter was hardcoded to something like 2 or 3 packets in the linux kernels. This was causing the initial page to require multiple RTTs to load so I figured I would change that and recompile the kernel to make sure all out pages would be transmitted in one go. This made out site perform a lot better compared to most sites at the time. Funny at the time I was a bit worried that the IETF would discover this and shut us down or something. These days that parameter is default to something like 10 and you can increase it with a config parameter.

Hats off. Great interface and i enjoyed making a few simple beats which is probably a very small part what the system is capable of. Thank you

I think this article is cherry picking here trying to make Canada look good compared to say the USA and other countries. ( im canadain btw ) I think it's easy to take in 28K refuges when the illegal load is very small in Canada, of something like 20K per year [1] Compare that to the USA that needs to deal with an illegal load of 20X that. [2] And for 2019 the USA load seems to be twice that of 2018. [3]

1. https://irb-cisr.gc.ca/en/statistics/Pages/Irregular-border-...

( and this is just the sw border, not accounting for flights ) 2. https://www.cbp.gov/newsroom/stats/sw-border-migration/fy-20...

3. https://www.cbp.gov/newsroom/stats/sw-border-migration

I suppose it depends if the use/need is in serial or in parallel.

So there are 3 basic cases that websites use:

1. Site and static content are served directly by your webserver. ( HTTP2/push helpful )

2. Site served by your web servers but static content is served by a CDN ( HTTP2/push NOT helpful )

3. Site and static content proxied by some service. ( HTTP2/push helpful )

If you agree that using a CDN for static content is a good idea, then it would seem HTTP/2 Push is useless. The website is served from your servers while the static content is served from a CDN so you can't "push" it in the same stream as you webpage content. Am I missing something here?

I'm not sure if I never understood this right, but it seemed server push was not an ideal overall solution. It seemed it was targeted to push needed css/js/image in the stream of the page result. But most assets that one would want loaded are coming via an external CDN so doing a push of that does not make sense. Yeah, I guess if you are proxying your whole site via the CDN. Am I missing something here?

I hear ya...of the two bolded feature headings in the story we get: 1. Supersonic speed ( always welcome ) 2. League of extraordinary emojis ( the last thing I would care about in a new os release )

You can get centimeter accuracy with gps. http://swift-nav.com/piksi.html Even with cheaper modules and phones that dont carrier phase rtk, you can get meter which would be plenty for this application.

Very rare data access is disjoint, unless you're only doing key/value put/get. I think the interest of Redis is that it has many other features than simply put/get, and all those sorts, diff, etc typically would work a set of data that is being written in.

For sure having multiple instances will help some of this, but adds more complexity. Do you have your app write to multiple instances, and then read low latency from one, and read high latency from another? Is that data now consistent? Do you setup Redis replication and make sure that works right and then read from different replicas? Or perhaps you engineer some queue that does not block writes, groups them together and writes to Redis in a separate thread. Then you have to maintain all this and make sure it's correct, back it up, what are the corner cases, failure modes, etc.

From my experience, if you want to engineer things well, you end up essentially building out the same sub systems that a larger db engine has. Say Innodb. I'm smart enough to know that I'm not smart enough to build a one off complex system more correctly than really smart people that have been iterating over many years and improving things on something like innodb.

There are very rare, very specific cases where I would use redis over something else if I was building something realtime, large and important.

Last time I used Redis I was surprised to determine to my surprise that Redis was single threaded. Of course I could have just RTFM but I assumed incorrectly.

This means that if you have part of your application that requires fast consistent GETs, and then another application does a slower SORT, UNION, DIFF, etc, on the same db or even other dbs on the same Redis server, EVERY other client request has to wait for this slower command to finish. http://redis.io/topics/latency

This is something that one really has to engineer around in order to use it in an environment that requires performance and consistent latency. In our case of 1000s req/s it was just unacceptable to have the latency be affected, sometimes by 10 times, by a slower command.

I do love all the sort, diff, union commands.

Your script name is tracker.js which is not the best name as adblockers/privacy blockers simply block any names with the word track and many other similar names. If you name your script something else that is not blocked it would enable it to run, and be valuable to see the errors that users get when they block other js on a page.

Is there an option/way to have "lables" in panes. Just like you have a info bar for windows, I would love to label my panes. Normally have a bunch of panes setup for logs, and it would be really nice to label them so i could tell what is in each pane.

Yeah I'm running a MBP 8.2 also (released February 24, 2011).

16GB ram

512GB intel ssd

1TB sata instead of cdrom

1GB AMD Radeon HD 6770M

I always got the next best mbp but since this one I thought I was always going backwards on certain things so I have not upgraded. So a 2.5 year old laptop still has these advantages that are important to me...

- non glossy display

- I have 1.5TB storage now ( and can go higher now ) compared to the max 1TB that is offered now.

- I'm still on snow leopard , primarily due to the multiple fullscreen idiocy in lion and mountain lion ( but that might change now in mavericks)

My only reason to upgrade is that I would like the fans to come on less when I'm plugged into an external 30" display. It's a little disappointing that the last 3 mbp releases are not, or making me thing twice, about hitting the buy button.

Decided to do a quick test..

//make yourself a file dd if=/dev/zero of=1m count=1 bs=1m

//post directly to origin

time curl -F "file=@1m" -X POST "http://up4.pinkbike.com/upload/t.php" -w %{speed_upload}Bytes/s

array(3) { ["HTTP_USER_AGENT"]=> string(81) "curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8y zlib/1.2.3" ["REQUEST_METHOD"]=> string(4) "POST" ["CONTENT_LENGTH"]=> string(7) "1048770" } 230177.000Bytes/s real 0m4.560s user 0m0.004s sys 0m0.010s

//post via Cloudfront to same origin

time curl -F "file=@1m" -X POST "http://dhima35gjf4ct.cloudfront.net/upload/t.php" -w %{speed_upload}Bytes/s

array(3) { ["HTTP_USER_AGENT"]=> string(17) "Amazon CloudFront" ["REQUEST_METHOD"]=> string(4) "POST" ["CONTENT_LENGTH"]=> string(7) "1048770" } 227055.000Bytes/s real 0m4.623s user 0m0.005s sys 0m0.011s

So upload time is not much different from my location. The origin is in San Jose, and in my case I'm in Vancouver BC, and going through the Seattle cloudfront edge. Would be interesting to see what you get from other locations ( I'm occasionally getting "ERROR: The request could not be satisfied" with cloudfront post )

Some more details here http://docs.aws.amazon.com/AmazonCloudFront/latest/Developer...

I'm curious if the edge receives the full POST/PUT first and then does a complete PUT/POST to the origin, or does it forward as it's receiving.

When we started we bought these boards http://www.supermicro.com/products/motherboard/QPI/5500/X8DT... and at first we outfitted them with one low end CPU and small amount of ram 6G as those were our needs and that's what we could afford at the time. 2 years ago we upgraded those machines with dual 5560 cpus, and 48Gram for not very much money, and in fact they run our production DBs right now. They still are very competitive if you stack them up to current e5 models. We added more servers last 2 years and they have been E3-1240 V2 based single cpu, 32G ram. You can't beat the price/performance there. So in 4 years we still have not obsoleted much but some older ram and base cpus.

KVM is really easy to setup. Install the package on your linux distro, start up virt-manager if you want gui, "start" new machine and install whatever you want from any cd image you have. Of course once you wrap your head around it you'll want to do it with cli tools and custom automate it. But basic virt-manager might take you a long way. Once you have multiple machines and you want to migrate between hosts you'll have to setup a shared storage. That can be as easy as an nfs share/mount. We started with just 1 ssd for that, but then built a dedicated box with many intel ssds on hardware raid 10. Never had an issue. But shared storage/live migration is not always needed and can add more risk. If you engineer it that all your hosts are independent and you have redundant services for everything, then you dont need to live migrate. If you need to free up that host, just turn it off, as you have redundant services running on other hosts.

In fact on our Dev systems we run KVM on our osx laptops nested in a vmware vm. ( vmware can nest like this passing hardware flags to the guest host ) So on osx you run vmware, which runs a linux vm, then that vm is used as a kvm host to run other vms via kvm. This way we can run exactly 100% the same image locally as is in production.

In fact if you really want to do some crazy plumbing... the VM host on my laptop has a VPN link to our DC, this puts it on the same internal network as our DC production hardware. I can then live migrate a production VM ( like a web front end ), onto my laptop, while it's fully operational doing processing for the production environment. On my laptop it will still be, via VPN, receiving and processing live web requests on our website, and properly sending back data to the proxy and user. Not very performant, but the flexible plumbing is nice if you want to test/debug a clone of the exact production system locally.

We actually went via Bandcon, which was then bought by highwinds. BW is around the going price $2.5/Gbps and it seemed to be level3 at the beginning and now it's seems more of a mix. ( I should specify that we have a gbps port but we only use about 100 mbps as it's only the html we serve from there ) We use another 2 Gbps of traffic via CDN for all the static/video content but that is of course a different cost ) But it's nice when the CDN ingest point is in the same physical DC as we are.

I just looked what 250TB would cost us on s3, $20k/month, or $240k/year. ( im not even counting the put/get usage )

You can build it, for ease of math, 100x 3TB seagate constellation. 100x $250 = $25k, another $5k easily covers a 45 jbod and raid card and server with ssd zil and arc for zfs and you're done. so $30k. Get 2 more for redundancy and backup as you see fit.

So over 3 years, 720k vs apples to apples 90k ( if you got 3 of those servers) so you save say $600k. You can get a decent remote dev for $200k/year for that time.

I wanted to throw my experience into the ring because there seems to be such a fear of colocation. We knew nothing about colocation and decided to build some supermicro servers ourselves and install them and a switch in a colo 4 years ago. I read the all stories "i had to get up in the middle of the night to drive to the colo. it was the worst move ever to colo", and they are total bull. Even the biggest noob can setup things so it's totally remote. Servers have a dedicated ipmi port ( remote console over ethernet ) that will make it as if you're sitting at the server remotely. You can even mount a cd/image on your laptop, remotely so the hardware thinks that cd is in that machine. Hell, I can reinstall the bios on the server remotely, OS, everything. Why on EARTH would you have to drive to your colo? You can get servers that have 4 ether ports that you can bond in pairs to different switches. You can have hardware raid so loosing 2 drives in a server is no big deal, and you can take care of it at a later time. We have drives fail sometimes, but things keep on ticking. With the costs you save you can have triple redundancy if you like, and the benefit of consistent latency and better performance always. We have 250TB of storage and its double redundancy and also a remote backup. It cost ONCE what we would have to pay for a few months on the cheapest storage service.

We run straight kvm virtualization on our own hosts for flexibility. We run dbs on bare metal. I hear all these stories of people vms "crashing" all the time but i can tell you we have only had 1 instance of a vm, or in this case host dying in 4 years. Happened to be one of the video conversion hosts that is pinned 24/7 and it turned out it just hit some un recoverable memory hardware error. No big deal there were others.

Flexibility? We can clone and spin up VMs at will. We can live migrate and upgrade hosts. We can automate things with virtlib to our hearts desire.

Costs? $1500/month for direct equinix colo ( includes power, full rack, and gigabit connection from tier1 provider ) Never had a power issue, never had a network issue. We also use a CDN for static stuff and thats extra. We started with 3 servers, now are at a dozen, and adding a new one does not add a new monthly expense.

You can have a E3-1240 V2 @ 3.40GHz server built for $1500 and that as a host can run most of our front end stack. Sure we have 6 of those for backend crap, redundancy, but we actually run most of our stack on 1 of them. Mostly we do that for shits and giggles, but also because the interaction between the www, redis, mcd, zeromq is a few ms faster when it does not go over physical net. So if you over optimize like us, and want 30ms page gen times, you can nerd out like that.

s6 CPU: 8 MEM: 32080MB total running CPU: 16 MEM: 16384MB r-fp1 running CPU: 2 MEM: 1024MB r-mcd2 running CPU: 2 MEM: 1024MB r-www2 running CPU: 2 MEM: 4096MB r-www3 running CPU: 2 MEM: 4096MB r-red1 running CPU: 2 MEM: 2048MB r-red2 running CPU: 2 MEM: 2048MB r-zmq running CPU: 2 MEM: 1024MB r-zmq2 running CPU: 2 MEM: 1024MB

front end proxy, www front ends, redis, zeromq, memcached, etc. Excluding mysql db which is on bare metal. This serves our site that handles about 200 page views per second peak day, and that is at 25% host utilization. Our pages generate ( no caching ), including redis, zmq, and maybe 25 db mysql calls per page in about 30ms. You can optimize things too like...you know that the default config on a server will kick down the cpu to 1.6Ghz if its not really loaded, and that means page gen times in our case would be 15ms slower. Hell, we dont have to try to save power, so we can kick that sucker to 3.4Ghz all the time and make sure users get the benefit of that. Nice to be in control of the host.

We never needed remote hands or anything like that, but that is available a phone call away. I visit the colo in San Jose once a year and I schedule it with my motocycle trip down there. Sometimes I just dust the servers off, pet them a little and look at the pretty lights.

Of course ec2 has it's use. If your html traffic spikes higher than 1 gbps, then it's nice to have the flexibility of a fatter distributed pipe. If you want to optimize for rtt then it's nice to be able to spin up in a different geographical areal.

I think what bugs me the most is that a lot of companies use the argument of, if you get high traffic, like slashdotted or hackernews you can spin up a 100 front ends easily and handle it. We've been on the top of hackernews and the change in traffic was in the noise floor as compared to 200 r/s we normally handle. The point I'm trying to make is that if you engineer your app better, and understand and fix issues with generating your pages faster, you wont need the fancy scale to 100 front ends bullshit. ( tip. it's probably your database queries anyways so optimize that. it's not the print/echo statement that is outputting html on the front end ) Of course some do require webscale and it's a good way to go with ec2 and all the extra costs and engineering, but it seems that every joe blow and his blog or app seems to think they need so spin up to 100 front ends.

Sorry for the rant. I actually think that ec2 and the likes are the future and as tech gets better and prices get better I can see it making sense for more and more. I just wanted to give a contrast with our current setup.

I know many services block or negatively rate connections coming from hosting providers. Personally we see that most of the spam attempts are via cheap VPSs. I'm not sure if they actually hack these or simply just purchase a $9/month host, install VPN, and go to work on spam. Our system keeps track of all netblocks and the companies they belong to and we classify those so our site can take action accordingly. Normally we just alert mods if a new user is created or a new first post is placed by a user originating from one of these hosting companies so it can be checked. Banning access totally from these would leviate some work. My personal experience has been that access to my site via VPN services or ips belonging to hosting providers, are mostly spammers. I think that eventually everything will go anonymous and that will make it much harder to detect and filter spam/fraud. It's going to take smarter tools and changes on sites to detect spam/fraud by usage patterns instead of source. The sad part is that these changes will effect all legit users negatively in order to try to prevent the few bad apples.

If you look at a site/product like Lumosity.com, you can compare your results to other age groups. If you accept that the tests on Lumosity do test particular brain functions/areas then its clear to see that from the millions of user that take part, the performance average does decline with age. It's actually quite dramatic to see that a 40 year old that scores in the 90% in a test category, fits in at 50% among the 20 year olds.

I added something similar to our framework where we do the encryption server side when a form is generated. In our token we encrypt a form generation time and captcha question and answer variables. This allows us to easily render on the form a textual or graphical captcha and pass the answer encrypted. The form processing simply decrypts the data and decides one, if a form is too fast or stale based on the difference of the form generation and submit time and two, it compares the captcha answer to that which was passed in the encrypted token.

Would this have helped how you feel better? Or would you feel the same way?

"This is retarded! There's no delay in the while loop. How many times does the client have to connect to the DB per second? Thousands! I can't believe this implementation is so fucking stupid!"

In this case we try to refer to the code directly, taking our reference to the person.

Author states that he is using innodb and insert delayed is a myisam mechanism. Of course in innodb the buffer pool / insert buffers can be thought of the same way. The delay the author is noting is probably due to flush of the log file data.

If you configure innodb correctly it comes down how much IO you can do on your log fs, to make sure you can sustain some update rate to this sequential log. Then nest is the IO to the db fs, so you can sustain the rate of the flushes. You can tune some of this much better with some percona mysql patches. Innodb does insert batching for you to minimize IO, and if you are doing updates to the same data it may even buffer the changes in buffer pool.

2 years ago we were discussing a few of the direct advantages of this in a comment here http://news.ycombinator.com/item?id=1143317 including tcp_slow_start_after_idle which also interacts with icwnd.

Also it's much easier as of late to get the benefit from a larger initial cwnd. Back then you needed to recompile the kernel with source tweaks, now you just use a backport or depending on your distro version you already have the benefit as kernel 2.6.39 has the change... http://kernelnewbies.org/Linux_2_6_39

But this is just a threshold issue. What does the difference in cost have to be for you to rent vs buy. Let's say it's $60 to buy a season of a show on hard media that you can watch as many times you want and yes, you own something that will collect dust for a life time. Perhaps you can resell it, but that just means your initial cost was lower and you in a sense rented it for the difference in price for that time period. Now if the cost to rent is say $20 for a season would that be worth hassle free experience of the content? How about $10, $5? At some threshold the value vs cost starts to make sense.

Hmm, well if the amount spent was the same it comes down who does more with the money. Where does the money go:

megaupload owners - fast cars, hookers and blow

studios - fast cars, hookers and blow

actors - fast cars, hookers and blow

apple - wages for workers in china.

I guess the lesser evil is the studios...at least they are employing local hookers!

The point I'm trying to make is....shifting the pain point. In this example itunes has been shifted past a particular persons pain (time/cost) point to be a convenient legal solution. Good point on the non availability in certain regions. At least my experience with itunes is that they are improving the service to lower that pain point in those areas too. You could not get certain content in canada a while ago, so the alternative was to find it on other sources, but as itunes adds more content in more other regions, this pain is reduced and it becomes the convenient service more and more.