The internet in general should return back to the old days when only somewhat educated and skilled people use it. Since it became mainstream because of smartphones things went downhill.
1 karma · joined October 31, 2020
Mainstream data scientists don't do data science, they are either ML engineers or data analysts who use ML Python libraries and promoted to data scientists with bigger paycheck. With machine learning and AI being the trend, the job is sexy and it is a chance for the company to market itself as it uses AI and cutting-edge tech. I worked for a data solutions company and while I work together with data scientists to propose a design to our clients, it was the AI part narrated by the data scientists that makes clients ready to throw money. Even if the requirements seem impossible or at least very difficult to achieve. These projects often fail because the data scientists couldn't reach the accuracy goal written in the contract and the project ends up in the trash. These data scientists eventually leave the company or get fired, only to find another job within a short time with even bigger salaries.
Now for the data engineering part; I wish the OP all the best with his career, but he is still in the honeymoon period and didn't witness the misery of being a data engineer.
- The job can get very repetitive very quickly, unless he'll work on infrastructure, his tasks will be mainly focused on maintaining existing ETL pipelines or building ones, and both are labor tasks. You'll end up a data plumber who makes sure data goes from A to B and then C and that's it. You'll seldom find something new or revolutionary to work on and you'll keep using the same tools as long as you're in the same company. Even when moving to a new job, you'll pretty much be hired because of your knowledge of the same data warehouse or ETL tool you used in the previous job.
- Data engineering is an underappreciated job. If things go right, nobody pats you in the shoulder. When shit gets loose, you'll be the one to clean up the mess. What makes it worse is, you can't leave this mess long because data is a snowball effect; if you leave it unprocessed, you'll end up with more data clogging your pipelines and what can be fixed within an hour can quickly take long nights and even days to resolve, and do you know what this means? Managers won't have their fancy dashboards updated and they'll start panicking.
- Data engineering is not rewarding. Again, you're doing your job. No one cares.
- Data engineering has nothing to show for. Yes, data is being crunched and processed and baked. It's the data analyst who builds the fancy dashboards for managers, and the data scientists who create fancy graphs for managers, and the ML engineers who create fancy products for managers. You're just a plumber who, instead of fixing toilets and sinks, fixes data pipelines.
- Just recently, data engineering salaries are rising thanks to low supply and higher demand thanks to better awareness from CTOs and heads of data about the importance of the role, but until a few years ago, they were paid less than a software engineer.
I came from a very bad background. Because of my "nationality" I can't work for most Tech companies (because US law), and I spent 10 years until I could settle into a country that I may call home but sadly I don't like to live in.
Relationships? Nothing serious and due to me focusing to much to survive (and my average looks) I never had a stable relationship or found a good partner.
Be grateful of what you have. Some people consider me lucky to live in several countries and seeing the world while looking for a place to settle in. So you are in good place. Just take some time off (I mean 3-6 months) and travel the world, learn a new hobby, take cooking lessons, attend more meetups to meet new people who can spice your life. The hub you live in can be dull so change where you live.