HNHacker News
TopNewBestAskShowJobs

marklit

2,705 karma · joined December 27, 2011

Big Data, AI, GIS & Networking Consultant.

Author and Technical Analyst of https://tech.marksblogg.com/

submissionscomments
marklit··on 1.1B Taxi Rides Using OmniSciDB and a MacBook Pro
Sorry, I just cleaned that paragraph up. Cheers for the heads up.
marklit··on Ask HN: How to summarize lots of short-term contracts in my resume?
I'd have it all under one listing with the job title you're looking to have for the next two years going forward.

Put a short list of some of your more well-known clients and then a list of technologies you've worked with that you suspect future clients will be willing to pay well for in the coming two years.

Three sentences, your future clients will thank you for not emailing them a wall of text.

I don't see CVs as needing to be detailed catalogs and LinkedIn and Blogs are better places for keyword stuffing.

marklit··on [dead]
I've avoided going down this rabbit hole. Staff costs can outweigh software and hardware in some cases. Hiring someone that knows kdb+/q could cost a fair bit more than someone who only has some PostgreSQL experience.
marklit··on [dead]
The benchmarks are aimed at OLAP not OLTP workloads.

If I had a large query set to run on each vendor I'd be more likely to hit compatibility issues which could mean fewer benchmarks going out. As it is I spend a lot of time getting hardware and software vendors together for these benchmarks.

If caches were being hit I'd expect a lot more DBs hitting single millisecond times in my benchmarks but as far as I can see, there is a clear delta between the various setups I've tested:

http://tech.marksblogg.com/benchmarks.html

marklit··on [dead]
Cheers, fixed.
marklit··on How we saved money by replacing Mixpanel with BigQuery and K8S
Does anyone have any experience of negotiating your costs down with Mixpanel? Years ago I remember my CTO ringing up our hosting provider and in the space of a phone call bringing some of our costs down 90%.
marklit··on Finance information on Google
Bottom right-hand corner of every chart has a "headlines" window. Might not be amazing but it is there.
marklit··on Show HN: CSVshare – Instantly share data with anyone and make it searchable
I just tried out the service. Some feedback:

1. It would be nice if I could upload CSVs that are gzip compressed.

2. When I download the CSVs it would be good if the filenames included gz at the end as of now they're written out as a series of random characters followed by a .csv extension.

3. Time columns have been transformed after I downloaded the file I just uploaded. The date is in US format and the time information is lost. 2010-12-01 13:51:00 became 12/1/10.

4. When trying to view a 50K row CSV on your site Chrome slows to a crawl.

5. I originally tried to upload a 1-million-row x 9 column, 33 MB, gzip compressed CSV. It would be good if the service limits could allow for something like this.

This service should do one better than just sending a CSV via email and having the other party opening it up in Excel.

marklit··on Windows 10 Fall Creators Update: the 10 best new features
I find the Windows 10 Task Manager very useful. Good to see GPUs will be listed as well now.
marklit··on Switzerland: How buying real-estate can kill you financially/reasons for stocks
Though IT support would count, I was thinking more along the lines of supporting traders, such as quants and the people that specialise in their toolkits that support them.
marklit··on Switzerland: How buying real-estate can kill you financially/reasons for stocks
I have to argue that there are both tax rates and job opportunities available in Switzerland that you'd be hard pressed to find elsewhere in Europe.

If someone it taking home ~300K CHF / year in some sort of support role in a bank in Zurich the above overheads don't seem to be such a deal breaker.

marklit··on Switzerland: How buying real-estate can kill you financially/reasons for stocks
Their assets in Switzerland would be priced in CHF. It could be a good diversification from USD-priced assets they have at home. There was a single-day 30% pop in CHF a few years back.
marklit··on Why PostgreSQL is better than MySQL
Does anyone have a link to a collective list of all known bugs in PostgreSQL?
marklit··on Switzerland: How buying real-estate can kill you financially/reasons for stocks
I would suspect a few of these wealthy foreigners buying property are trying to make sure there wealth is tied up in something their home state cannot take away ownership of. If it depreciates a bit, it's probably fine. 97% of something is better than nothing. I also suspect they have nice stock portfolios already as well.
marklit··on Next-Generation GPU-Powered EC2 Instances (G3)
Anyone care to chime in on why spot instances are now 10x on-demand instances? I've got a thread going here: https://news.ycombinator.com/item?id=14769026
marklit··on Ask HN: Why are p2.16xlarge EC2 spot instances 10x the price of on-demand?
I'm wondering if the Cloud providers are seeing cryptocoin mining as the primary workload at the moment and don't want to be left with a ton of inventory the next time there is a difficulty increment and clients move on.
marklit··on BigQuery vs. Redshift vs. Athena simple benchmark
If you mean not as a cluster Spark standalone can do conversions.
marklit··on Compiling MapD's Source Code
I don't work for MapD. I run my own consulting company in London and all of my clients are British firms and/or have a presence in the UK. MapD doesn't have a UK LTD setup or anything of that kind.

My Bio and CV are right at the side of every blog post, how hard is it to read them?

marklit··on 1.1B taxi rides benchmarked on distributed GPU-powered MapD
Same dataset? Mind telling me the uncompressed byte count?
marklit··on High-performance .NET by example: Filtering bot traffic
I've seen bot traffic claiming to be recent versions of Firefox from residential IPs in the Ukraine pulling robots.txt. Sometimes this is one of the few clues to go on.
marklit··on Ask HN: BigQuery vs. Redshift vs. Athena vs. Snowflake
I've used BQ, RS and Athena. Their benchmark results and links to how I conducted benchmarks on each can be found here: http://tech.marksblogg.com/benchmarks.html

I could probably write a book on my experiences but I'll keep this short.

BigQuery requires the least amount of work to keep performant but this comes at a cost (this cost matches Athena).

Redshift seems to have support for just about every BI tool out there including Periscope which I've found to be popular with my customers. The downside is that you've really got to think about how you layout your data, how you write your queries and there is something of a weekly task of bringing up the console and hunting down the worst-offending queries for optimisation.

Keep in mind if you're already on Google Cloud or AWS you could incur bandwidth charges moving data to the other provider.

marklit··on Detecting Bots in Apache and Nginx Logs Using Python
Thanks for the heads up, I've updated the gist.
marklit··on Alenka: GPU database engine
If anyone is interested in getting this setup I put together some install notes along with some speed bumps I encountered while benchmarking Alenka a few months back: http://tech.marksblogg.com/alenka-open-source-gpu-database.h...
marklit··on Show HN: GUI for writing Elasticsearch queries
I'm a fan of writing out queries using SQL. Here's a handy plugin to do that with Elasticsearch: https://github.com/NLPchina/elasticsearch-sql
marklit··on Ask HN: Digital Nomads Who Stopped Wandering- Where Did You Settle?
Tallinn. Cheap rents, free public transport including boat rides to the nearby islands, not far from the UK if I need to see clients onsite and 500 Mbit/s Internet for €37 / month.
marklit··on 1.1 Billion Taxi Rides with MapD & 8 Nvidia Pascal Titan Xs
Todd Schneider already did a bunch of analysis on the data. I simply wanted to use the dataset to compare various OLAP engines.

http://toddwschneider.com/posts/analyzing-1-1-billion-nyc-ta...

marklit··on Brexit: Berlin eyes Britain’s tech talent
At £43,0001 / year in the UK you'll take home 75% of that. https://listentotaxman.com/43001
marklit··on The Estonian president wants a more digitally-integrated Europe
It might not be mentioned in the article but a lot of online retailers, including with 50% of the purchases I try to make on Amazon.co.uk, refuse to ship to Estonia. Even digital good can often be difficult to purchase.
marklit··on Ask HN: How common is it to plug full time gaps with contractors?
I've seen it happen a few times a year with my clients in London. Occasionally the contractors become part of the furniture though as good candidates that are available at the right price are not always easy to come across.
marklit··on Ask HN: Contractor rates and Full-time salaries in UK outside London?
I've been contracting in London half the year and remotely from Estonia the other half of the year for about three years now. It's hard to get UK clients to agree to remote work. It usually takes 1 - 3 months for me to find a remote contract but the lower cost of living and high quality of life here is worth it. I've had to knock £70 off my day rate to keep my clients from considering other candidates.

I don't see a lot of contracts from outside London appear where the day rates are anywhere near what clients inside the M25 offer.

← PreviousPage 3 of 5Next →