olavgg

The biggest advantage of ZFS from a operational experience, is that when you have problems, ZFS tells you why. Checksum errors? Something wrong with the hard drive or SATA/SAS cables. Is the disk slow, zfs events will tell you that it spent more than 5 seconds to read sector x from disk /dev/sdf. The zfs cli commands are super-intuitive, and makes fully sense. Compared to ie. virsh, which is just weird for managing vm's.

It definitely worth the hassle. But if everything works fine for you now, don't bother. ZFS is not going away and you can learn it later.

ZFS snapshot, send, receive, clone, spin up another postgresql server on the backup server, take full backup on that clone once per week

I actually always build PostgreSQL from source as I want 32kb block size as default. It makes ZFS compression more awesome.

BAR is a really great RTS game. By far the best one today. It is very balanced and has a lot of depth. Highly recommended for any RTS enthusiast.

It’s been 17 years since I got my first Linux job in 2008. Where I live, that’s rare, 99% of the industry here is a 'Microsoft Shop,' and the biggest player in town is practically married to them.

I started out at a small Linux company working with Plone CMS. The pay wasn’t great, but it was the perfect place to learn Linux and Python. Since then, I’ve used Linux every single day, become a Java developer, and started a few businesses. Using Linux of course.

But lately, things are changing. Companies are realizing that when it comes to Data Engineering and Science, C# just can't compete with Python's ecosystem. Now that they need to pivot, they're looking for help, and there are very few people in this area with the experience to answer that call.

Do not buy Kioxia. If your drives dies after a few years, but can come back to life with an firmware update. They will not give it to you unless you have a support contract.

Solidigm have all their firmware available for everyone their website.

As a startup, what is it in for me to switch from Java, Spring Boot, Hibernate, Beam, Flink, Pulsar, Vault, KeyCloak ecosystem to C#.Net? Is the documentation better? Do I get better performance? Is the community larger and more stable?

A standalone Pulsar, is actually a great way to learn Pulsar. It is one command to get started: bin/pulsar standalone

It can also be used in production. You do not have to build a distributed Pulsar cluster immediately. I have multiple projects running on a standalone Pulsar cluster, because its easy to setup and requires almost no maintenance. Doing it that way makes compliance requirements for isolation simpler and with less fights. Everyone understands host/vm isolation, few understand Pulsar Tenant isolation.

If you want a distributed Apache Pulsar cluster, be prepared to work for that. We run a cluster on bare metal. We considered Kubernetes, but performance was lacking. We are not Kubernetes experts.

The biggest reason is that you may also have other consumers than just Clickhouse.

I'm in my 40s and run my own company. We deliver a data platform, our customers can choose between our self-hosted solution or run it on AWS/Azure for 10x higher cost.

SeaweedFS has evolved a lot the last few years, with RDMA support and EC.

TrackerFF sounds like someone on the left trying to defend the wealth tax. As a Norwegian who believes in freedom and the right for anyone to start their own business, I think the wealth tax is one of the most toxic taxes ever introduced. In practice, it makes Norwegian-owned companies about 30% less competitive than foreign-owned ones.

It’s sold as a “tax on the rich,” but in reality it’s a 2.1% annual tax on businesses. The companies have to cover these costs, and the result is that everyone — rich or poor — ends up paying for it. The only winner here is the state, which wastes the money on useless projects and subsidies.

It gets even dumber: businesses that invest in emergency preparedness — like storing gravel or materials for war or disaster scenarios — actually get punished by the wealth tax. Imagine paying 2.1% tax every year on a gravel pile. No, this is not a joke: link: https://www.dn.no/innlegg/beredskap/sikkerhetspolitikk/formu...

The consequences are clear: more than half of the 400 wealthiest people in Norway have already left the country. There’s no risk capital left for startups, and outside the oil and gas sector, businesses are struggling. Plenty of smart young engineers fresh out of university can’t find work, and hiring of junior software developers has basically stopped.

When we talk about taxes, we should be talking about incentives and motivation. If you tax people so hard that they lose the drive to work and create, then you’ve got a real problem.

If I had the chance, I’d move to Sweden or Switzerland immediately. But I can’t, because I’ve got a family to take care of here.

Mattermost is open source, but the licensing is complex and full of bullshit. It is not a community driven project. Once you have installed the self hosted solution, you get a user interface that asks you to upgrade to the enterprise edition in every corner and menu.

A self hosted version is better than nothing though.

The last 15 years, servers has gone from 3x memory channels to 12x, while desktop still only have 2x memory channels. It is by far the biggest bottleneck today.

Probably by using partitioning.

I love Debian, but it does have some weaknesses. For example with virtualization, when you enable SR-IOV, apparmor goes bananas. With AlmaLinux + SELinux there are no problems. I use both Debian and AlmaLinux on my servers, and with that combo I feel I get the best of the best. But I think AlmaLinux is more polished and that SELinux is superior to apparmor.

If you have many beans that does IO, startup will be slower. For example, just removing sql/database initialization can shave off a second.

Before I knew about Keycloak, I need to figure out how to use Spring Boot to authenticate via Azure Entra Id. I could't use Spring Boot Security OAuth2 as I couldn't figure out how to bind Entra ID groups to roles in Spring Boot. I saw a great video from Okta where they broke down all details down to each http request (don't remember the link to the video), and then implement each http request/redirects to Entra ID. Finally I got the token and could then use the Graph API to get group memberships for binding a Spring Boot role.

I still used Spring Sessions though, where a successfull authed user got a new Spring Session. The reason was that I liked the idea of having beans with session scope, for example where each user/role has access to a specific database schema.

I disagree, the wealth tax is the issue. The reason the wealth tax is an issue, is that making investments becomes a burden. Norwegian owners have to pay 2.1% of the company value every year as a tax, and when you invest, it can take 5-10 years before you see profits. The calculation is like this: your company is worth 100 million. Your result this year was 0. No gains nor any loss. You still have to pay the wealth tax, and you need to take that money from somewhere. The wealth tax is 1.1% on personal assets. Which means you need to pay 1.1 million. You have to sell a share of your company assets to someone else. You will not get market rate, as you are the initiator for the transaction. We assume you sell assets worth 3 million for just 2.1 million. Now your company has a positive result of 2.1 million. Next is the company tax, 22% which means you have 1.638 million left to pay dividend. The dividend tax is 37.84% which means you as a person now get 1 million to pay the wealth tax.

The exit tax closed a loophole, I do agree that the exit tax needs some adjustments though.

Cognite built their dataplatform with foundationdb.

As a Norwegian, I can say that living here is awesome. Unless, you're a business owner, a founder, or have no work experience. The effects of the wealth tax has been insane. Businesses no longer have money to invest and every business that is not state-owned, is in survival mode.

The majority of the "rich" have already moved to another country. Business owners have to sell their business to a foreigner to be able to pay the wealth tax. These business owners use this money to move to another country and retire. Businesses has stopped hiring people without work experience. Even if you have a computer science / engineering degree. The brain drain has already started. The rich no longer invest in local initiatives like local sports, local businesses or volunteering as they are "disconnected". A lot less tax is being paid and to cover for this we need to withdraw more from the oil fund. It may take a few decades to empty the fund. But I think that money will be wasted within a decade if "hate" against rich continues. https://x.com/elonmusk/status/1857694219244048691

I am seriously worried about the future, and I am considering moving to Sweden. I love building businesses, but I feel that the state is stealing resources I need to build a business that can pay tax for the next generations.

The high sick leave thing is another serious matter. But when politicians deflect responsibility when they've made a mistake, especially when public scrutiny or pressure increases, they send a message to the rest of the Norwegian population that it is fine to take a few months off if work is too hard. This has not happened once, this has happened a few dozen times just the last few years. And founders, they do not have the right for a single day of paid sick leave unless they employ themself and pay extra tax.

You do, the normal way is to create a special metadata devices of mirrored SSD's. You do not need a 3 way mirror, as ZFS by default stores 2 copies of your metadata.

I would not call HN a bubble, Enterprises often have unqualified people making "expensive" decisions.

I experimented with spindowns, but the fact is, many applications needs to write to disk several times per minute. Because of this I only use SSD's now. Archived files are moved to the Cloud. I think Google Disk is one of the best alternatives out there, as it has true data streaming built in the MacOS or Windows clients. It feels like an external hard drive.

There is a company delivering a data platform for the Industry 4.0 named Cognite based in Oslo, Norway that migrated from Google BigQuery to their own database on on top of FoundationDB.

The video about it is available here: https://2023.javazone.no/program/85eae038-49b5-4f32-83c6-077... After watching, my thoughts were; why didn't you just use Clickhouse?

And no ECC memory?

Valkey has RDMA support, which offers significant performance improvements.

How many of the Apache Kafka issues are adressed by switching to Apache Pulsar?

I skipped learning Kafka, and jumped right into Pulsar. It works great for our use case. No complaints. But I wonder why so few use it?

Actually this is not important. Because they need to make a living, and with that they have to make some functonality of ClickHouse propritary.

You can scale the pure open source project really far. And if you need more, you do have money to pay for it.

People need to stop thinking that open source is free leech. Open source is about sharing knowledge and building trust between each other. But we are still living in a business world where we are competing.

That is true, though the implementation is weird.

I mount my nfs shares like this: sudo mount -t nfs -o nolocks -o resvport 192.168.1.1:/tank/data /mnt/data

-o nolocks Disables file locking on the mounted share. Useful if the NFS server or client does not support locking, or if there are issues with lock daemons. On macOS, this is often necessary because lockd can be flaky.

-o resvport Tells the NFS client to use a reserved port (<1024) for the connection. Some NFS servers (like some Linux configurations or *BSDs with stricter security) only accept requests from clients using reserved ports (for authentication purposes).