proverbialbunny

That's exactly why the quality has gone down, due to these speed subs. Back in the day speed subs were not a thing. They popped up when legal subs started popping up, almost as a way to compete.

I think the first big name speed sub group was horriblesubs, and they still exist today. Today fansubs are dominated by these groups. They're quality is okay to good. Sometimes they're the only option now, so I'd say the quality has gone down.

Back in the 4:3 days when anime was vbr and sources were interlaced fansub groups would go through frame by frame doing pulldown and denoising. It was amazing. They'd have high quality karaoke in the opening and ending scenes. When's the last time you've seen high quality karaoke on fansubs?

No, quality has gone down. They don't even do color correction today. Maybe in the last 3 years or so fansub quality has gone up a bit, but it's not a lot.. not enough for me to notice much.

>speaking as an amateur translator myself I always think it's good for the viewer to have alternative interpretations for more difficult-to-translate passages.

Yah. You don't often see translation notes any more either.

https://stackoverflow.com/questions/2550281/floating-point-v...

Floats were not historically adopted for speed, but because you can get more decimal places with small numbers like 8 bit numbers.

Today with 64 and 128 bit numbers, floating point loses most of its historical benefit, but of course it still has its benefits.

Most of them had codes you'd type in that was a save point. Many had actual save states on the cartridge.

The earlier games got around not having a code by having a secret early on the game that was revealed later on in the game so you could skip all of the parts you previously played.

I'd share public sites, but I'm not sure if it breaks the rules.

That may be part of the problem with the industry. The average viewer is none the wiser, so the translation industry gets to make a subpar product and get away with it. If it wasn't for fansubs, they'd have a monopoly. Thankfully, the industry has spent the last 10+ years competing with fan subs instead of outright trying to nuke them and it shows. Subs (and dubs) were far far worse not that long ago.

Ironically or perhaps paradoxically, as professional made dubs and subs have increased in quality fansubs have decreased in quality. It's almost like fansubs are giving the industry a push forward.

So, eg, if it's generic torrent tracker software, it's fine, because anyone can use it in a legitimate way (though it helps if someone actually is using it in a legitimate way), but if this is just for the site nyaa.si and no other trackers, they might lose this case if it goes to court.

I think all the games I had were 60+ hours, but I just liked JRPGs more than most genres.

FFVIII was picked not only because it was one of my favorites (FF1 too, SMB3, ...), because the remake today has turned it into 3 games, and even then those 3 games seem long by today's standards.

FFVII is still one of my all time favorites and is a good example of that. I think the first real fast game I played through was Metal Gear Solid. I'd rent, then buy if I liked it back then, and I played through all of Metal Gear Solid in something like 2 or 3 days. It was the first game I enjoyed that I did not buy, because I beat it while renting it.

But for actual simulating trading where calculastions compounds on itself, instead of one algorithm that calculates something and then is done, floating point becomes an issue.

----

Because of the downvotes, let's take a step back to 101 about floating point error:

>>> 1.20 - 1.00

> 0.199999999999999996

In this example, you're a penny off. This is a single equation. So you have to check for rounding error and possibly round up for EVERY calculation you do, which is time consuming.

Alternative, there is fixed precision types which are fast, very fast, faster than regularly checking for errors.

If you have an equation that calculates once and then is done, a penny or two off is no big deal, but when you're backtesting, a penny or two off for every trade compounds and you end up with dollars a day off. If the algorithm is identifying to the faction of a penny in the middle of the trading day and adjusting its configuration accordingly, it will be wrong, which creates a butterfly effect that ripples out into the day, and with high frequency trading it's somewhat uncommon, but possible, to get 10% off a day from floating point precision due to the actual algorithm failing on deciding the correct path forward while trading significantly compounding the issue. Now take that and compound it out months and you see the issue.

>THE COURT CASE IS LINKED IN TFA

If you look at the court case the only thing in it that is not hearsay is a dented bumper. Furthermore, she was dismissed. Because of a lack of evidence it never would have held a candle in court because it was all hearsay.

This is the problem with articles like this. They defame people, and the average person who has no legal background eats it up as the wholesale truth.

>I'm not sure why you're so insistent on defending if not a sexual predator then at least someone with highly questionable judgement.

Woo.. there is nothing in there that says she is a sexual predator.

>And again, maybe all of this has little to do with the whistleblower stuff, but there is no question about whether or not it's true that she had an inappropriate extramartial affair with a student, stalked him after he tried to end it and was fired over the whole ordeal.

It has nothing to do with the whistleblower stuff. It has to do with how garbage the article is. Falling for it is embarrassing. The fact that articles like these today are legal, is a sad state of affairs. In the US up until the 80s, articles like these would be against the law for hopefully obvious reason.

Yes, because over time thousands of multiplies and divides means you will get more than a penny off. The more high frequency the more floats become a problem.

>any relationship between a teacher and their student is non-consensual

That's because teachers teach students typically under the age of 18. Professors teach adults. She obviously wasn't a high school teacher. Some universities have no policy against professor student relationships, though in the last 10 years that has quickly been changing.

If it happened it was a relationship between two consenting adults, so there is not really a reason for them to care.

Getting fired from a university, getting called in to a psych ward to make sure she was okay, and then immediately after being let go is pretty irrelevant to the case. I'm no lawyer but I'm pretty sure this would be objected as irrelevant if anyone tried to use it in the court case. For all we know it may even be defamation.

If the point is to identify the speed (or ram consumption) of algorithm, then why not check for that itself instead of the vocabulary in an interview? Why be pedantic when you can instead measure how well they would do as a developer? In an interview you can ask followup questions to see how precise their ability to explain their thought process is.

If someone is so pedantic that they would consider the interviewee to have shot themselves in the foot because they said "The big O is n squared." without any followup questions from the interviewer, that doesn't sound healthy to me. I would worry this kind of culture would extend past the interview and it wouldn't be an enjoyable place to work.

Can you imagine working in a place where people regularly argue over terminology instead of just making sure everyone is on the same page?

(Full warning: I'm not a dev, so I'm coming in from the view of another industry.)

I've had projects where the model doesn't perform as intended. Because one person was making the model and another productionizing it, it was hard to identify where the performance difference was coming from. Was the bug in the model itself or in the productionization process itself? It took longer to figure it out than writing the model or productionizing it the first time.

It takes so long to deal with these bugs because the model gets changed, so then prod gets changed to match it. Changing prod (rewriting functions) has the potential to create a new bug, so you solved one but added another, and still can't identify if it is in the initial model or from prod. This continues over and over again, problem after problem.

It's noteworthy to mention if one person is doing both the model building and converting to production this problem is significantly reduced, but is still a problem. The problem is exasperated from the lack of domain knowledge, being that both people are in the dark from the other person's process.

Furthermore, what if you need to update the model? Do you rewrite prod doubling or tripling your work? Do you take that risk to introduce another potential hard to diagnose bug, even if you're the one doing both roles?

Or do you automate the process, so the same code being developed on is the same code running on the server at the end of the day? No more bugs, half to 1/3rd the amount of the work. Why not do it this way? It's soo much easier to debug a problem in prod this way. You can take the log data and spit it into the local machine and know what you're seeing is what the user saw. No more guessing where the problem is.

One way to think of it is software engineers would think it is absurd to write their code, then hand it off to someone who doesn't completely understand it, to rewrite it in another language and put it up on a server. "Why would you ever want to do that?" they would think, and I agree with this sentiment. It is absurd to have someone (even you) rewrite your work unless you have no other option, and you do have other options. Transpilers are a thing if prod needs to be in another language. I've written models that have to go onto embedded environments. I know these challenges all too well.

>I'd imagine many data scientists want to publish their work as python packages or libraries during their PhD, so they should be familiar with writing classes or functions that work at a bare minimum.

It depends if you're writing a library, like doing ML / machine learning engineer type work, or you're solving a domain challenge and writing an end to end solution for that problem, and are using standard cookie cutter ML for your phd, aka data science type work.

One leads to an engineering role, and not surprisingly writing a library for it is ideal, so other people can use it. Another leads to a data science type role and not surprisingly showing code snippets in your paper with plots / EDA and all, the same way you'd write a notebook at work, is ideal.

I'm a data scientist, not an ML specialist (though I have invented a new form of ML for work once, but that was just once and not my primary thing). I specialize in end-to-end domain problems I'm solving. I'll write a notebook to solve it, not that I have to. I've been in the industry longer than notebooks were a thing, so I'm fine doing it the old fashioned way. What I am not is an MLE. I don't need to write libraries for other users to use. I don't need to write custom ML. I don't need to do that engineering bit. To be fair, I have, and I know when it's the right tool for the job. On stackoverflow all of my points come from helping people with the glue parts between C++ and R, so they too can write libraries for R. I'm proficient in modern C++ too. I can do the library ML type work, and I have enjoyed it, but I really do enjoy solving domain problems more, so it's what I'm doing, and it's what the previous comments in this chain you're responding to are all about.

It depends on the company. Traditionally, yes, but deployment into production can be automated, so typically today it is something different.

An MLE is someone who specializes in Tensorflow or PyTorch. They write deep neural networks, reinforcement learning, and more. Often times the data scientist will make a model, specializing in feature engineering and domain experience, and use a generic ML like a generic DNN or xgboost or whatever it may be. It then gets handed off to an MLE who writes ML specific for the problem to get every last drop of accuracy out of the model. They then hand it off to prod. I don't think they're on call (I could be wrong on this.) so today they're not really deploying models much. They're more an inbetween.

I work at small companies and startups so I've never worked with an MLE, but I do have friends who are managers at Google who told me about it, so that's where this information is coming from, telephone game. In other words, I'd take this with a grain of salt. ymmv.

Starting in 2018 big name companies couldn't get enough MLEs and they pay higher than DS', but many bootcamps and universities center around ML skills, so companies started renaming MLE positions to DS positions. This way they get more applicants and they pay them lower. Win-win for them. Too bad it messes up the industry. Today about 1 in 3 data science jobs are ML heavy. They may be MLE exclusive or a hybrid wearing multiple hats light DS to light MLE type jobs.

You can identify which is which if they give you a white board coding problem. Traditional data science work will never have a white board problem.

>So thank you again for your responses, they are very helpful.

You're very welcome. I hope it helps.

I'm going to answer both questions just in case:

To become a data engineer / infrastructure engineer, there are multiple paths forward. I recommend doing BI work aka Business Intelligence Analyst Engineer. It's typically Tableau related work, so making dashboards and reports for management. It's still a data related role so you should feel comfortable and at home. However, it is also an engineering role. If you're the first BI at a company you'll often find yourself setting up an SQL server and doing certain data engineer-light type work to get data into the server. You'll need to set all of this up, so BI is a blend of data engineering and data analyst type work.

Once you've gotten familiar with BI work it's very easy to transfer to data engineer / infrastructure engineer type work. This is especially true if you end up setting up a data warehouse (as an alternative to MySQL) or data lake on AWS to do your BI work. You don't have to, but if you go that far, you're pretty much doing data engineering at that point. The line between the two is fuzzy. Data engineers and infrastructure engineers are expected to be architects, and by that I mean they are expected to future proof the schema of the SQL server / data warehouse (future proof setting up the database for new data so it doesn't become a mess). A BI is not expected to be an architect and imo the only way to gain that skill is through first hand experience playing with databases, so a BI is a good way to get that experience.

At the current company I'm at the infrastructure engineers are expected to do BI work. This is unusual as the data scientists typically do it (roughly 60% of data scientists do BI work), because one of the data engineers I currently work with was a BI at his previous job. (He was on the sales team, helping them with more than just dashboards, like helping with their Excel spreadsheet algorithms and what not.)

I'm sure others could paint another path forward. Data engineers are highly in demand so it could be as simple as applying. If you can pass a white board interview (leetcode style interview) you can skip this step and dive right on in. Just like any technical white collar job, you're expected to self-learn what is required for the job before going in, so absolutely read guides / take classes / read books / etc on the topic to learn more.

----

To get a job as a data scientist:

BI work is a good bridge too, but not in learning setting up database skills, but creating dashboard skills. Around 60% of data scientists in the industry do BI work. Me, I've had to create internal dashboards for diagnosing problems, so it streams in live data in a visual way. This is not BI work, but there is clearly a bridge between BI dashboards and internal diagnostics dashboards.

Technically a data scientist is a kind of data analyst so many people go from data analyst directly to data scientist. Around 30% or so of data scientists do only data analyst work but have the data science title. (This 30% number is a bit of an estimate.) It's that strong of an overlap.

Data scientists tend to specialize. There sales data scientist, a marketing data scientist, an engineering data scientist, ops data science, and so on. Often times, but not always, they sit on the team they specialize in, instead of on a data science / data analyst team. At smaller companies they tend to hire a data scientist and expect them to do one kind of role. So it comes down to what kind of data science work you want to do. Sales data science roles tend to be BI heavy. Marketing data science roles tend to be data analyst heavy. Engineering data science roles tend to be the heavy model building roles that are the most challenging out of the bunch. Ops data scientists tend to specialize in malware detection and self-reporting. Eg, if someone is hacking the company's servers, they might get notified of an alert, and then they analyze it and report on it. There are other kinds of data scientists, like ones at super market companies and restaurants that specialize in forecasting warehouse items.

Me, I'm a specialist that specializes in robotics and sensor analysis. I'm not going to lie, it's probably the hardest out of every kind of data science role. It's very heavy on the engineering side, not data engineering, but software engineering, because there is a lot of advanced feature engineering.

Most feature engineering is simple stuff like deleting missing values, performing the medium over the dataset or other kinds of cleaning and minor modifications like normalizing the data. Then it gets spit into an ml library that identifies the pattern in the data, so when new data comes in it can identify if it recognizes that pattern. Each pattern is called a category and most ML work is categorization, so maybe you're categorizing different kinds of customers and if you can identify a pattern in their shopping habits, you might be able to predict what they will buy next.

Advanced feature engineering might need to be used when your patterns are so complex ML can't pattern match it well, so you have to give it a helping hand and manually do some of the pattern matching. I've also had to invent new forms of ML too, but it's been a while since I've had to go that far. What I do is the farthest from normal for data science.

Most data scientists do not know advanced feature engineering, but it's one of the bridges between software engineering and data science, so leveling up software engineering can help on that front. (Which is also why I bring it up.)

A data scientist shouldn't be expected to know much or any data engineering skills. Instead, gaining managing upward skills helps. How to do an sql query to get data and how to write a join is enough. You should do fine, just try learning data science itself instead of learning data engineering, unless you're curious. (A lot of universities and bootcamps teach machine learning engineer skills and call it data science. If it doesn't have data cleaning and feature engineer, it's probably not data science. Likewise if it has tensorflow or pytorch in it, it's not data science.)

I tend to be seen as a product lead / owner / stakeholder, so I feel like I'm being called out. lol

I think one difference is the software engineers see me as someone who is helping them by making their life easier. I'm not just throwing work at them blindly. I'm working with them. Also, they like it when I include them in the data science brainstorming sessions to solve difficult problems. I guess it's seen as exotic or something, but whatever the reason, they really love to be apart of it.

In the 90s it was normal for truck drivers to literally drive 100mph in some parts of the country (No, I'm not making this up. A few states even had no speed limit on their freeway and semis would drive even faster.), but once gas prices rised during the bush administration most companies capped their driver's speed to 62mph to increase mpg.

During this time when gas prices shot up people stopped traveling for vacations as much. Ride share programs popped up at this time too. This was the time I got my first remote job, but that may be a coincidence.

On the quant side bonuses are distributed to the team.

You would be surprised. The second you use pandas with a custom data type (let alone any other library you'd want to use) it can randomly auto convert it to a float. Furthermore identifying when it randomly converts the type on you is a pain.

>so floating point basically must be used.

Quants tend to use fixed precision types. It is like a float in every way, except base 10 instead of base 2 so there is no floating point error.

This is great for making a dashboard, a report, or some other kind of analytics, but when it comes to a service the customer uses, you typically never want to load the whole notebook. This is where the industry standard way of loading the whole notebook tends to fall on its face.

What we do is the cells that will end up in prod are written as functions inside of the notebook. This helps reduce globals when writing the notebook, so it is good form when prototyping, but also it allows just those functions to be called from the notebook, instead of running the entire notebook.

You will probably want to write your own library to do this, but in the mean time there is one that works for this purpose https://github.com/grst/nbimporter (Ironically the author doesn't recognize this use case.)

Using nbimporter you can import a notebook without loading it. You can then call functions within that notebook and only those functions get loaded and called.

In my notebooks I have a process function which is like main(), but for for feature engineering. On the prod side the process function is called from the notebook. Process calls all of the necessary cells/functions for me in the correct order. This way the py wrapper only has to call one function, then the ML predict function gets called, so it's pretty small on the .py wrapper side. There are tests written on the .py side, IO functions and what not too.

Data engineers love their classes, so it's easy to write a class that calls the notebook, and best of all calling a single function this way does not load globals, so the data engineers are happy. It's a nice library, because otherwisw you'd have to write your own (which you may end up wanting to do).

This way if the model doesn't work as intended in production it's my fault. We log everything, so I can run the instance prod caught on my local machine, figure out what is going on, update the model, and then it can be deployed instantly.

Version numbers on the engineering side I can't comment on as they have their own method, but on my end the second the model writes to a database then I strongly push for having a version number column or a version number metadata table in the database, so it's easy for me to access for future analysis.