Warner Bros Suspends Sales of Batman: Arkham Knight PC Version
community.wbgames.com
community.wbgames.com
The average rating for the game on Steam was 31%. Since this was the first AAA game to be released on Steam after the release of Steam Refunds, it's also suspected that a high number of returns could force drastic action.
QA/QC should not happen at the customer level.
This is an interesting comment because the primary suspected reason for the implementation of Steam Refunds was Steam Greenlight, which allows anyone to upload a game to Steam, and after community vetting, the game could be released on the store proper.
Of course, many of the games were either poorly tested or use assets which the developers claim as original but are not. Steam Refunds fixes this in most cases
Jim Sterling has a good showcase of these games: https://www.youtube.com/playlist?list=PLlRceUcRZcK17mlpIEPsV...
The point of the unit test is to test the interface in isolation, not the implementation - e.g. do you really expect when testing a hash-map the elements to be in specific order, or do you test for their presence?
The metatesting reason for this confusion is that in the example you used, the scope of unit testing is the same as the scope of integration testing. If a class has 0 dependencies, a unit test is also an integration test, because it tests all the dependencies of the class, of which there are none.
So going back to your original point, your affirmation is true for full integration tests, i.e. tests that do not stub any dependency. If you do not stub your network connection to the memcached server, the same test - albeit with a different setup - can be used for a local hash table implementation.
> If you have an unit test on private members, then you can't change the implementation safely without breaking tests.
This makes absolutely no sense
You change the private implementation, you change the unit test. It's that simple
Then you keep your API the same so that external users don't break.
It seems to be from the same people that like to whine about "missing tests and lack of coverage" quite funnily. It seems they like to nitpick and idolize tests instead of shipping
In short, you are testing the interface of that private method, not the implementation.
But this is what I seems to be getting out of it: You are given a black box, with inputs and outputs. There is also a spec (it could be in your head for all I know) that defines that for certain inputs, certain outpus are expected. This spec also tries to cover quite a lot distinct cases. Each such representative case of input and output is an unit test. (If your spec was really in your head, your unit tests kind of becomes it, or I like to think about it in this way - a Unit Spec :)).
The tricky part is when this blackbox is internally working with other blackboxes. Unit testing is all about testing the blackbox in isolation from other blackboxes. As such one needs to isolate them away. Currently what I'm using is DI (Dependency Injection) with guice/gin/dagger to achieve that.
Thanks for all comments, it seems I have to fill my gaps in what I know.
It's also a matter of praticality - I simply odn't have time to write tests for each and every method.
To counter that I'd say you should have a test case to cover the leap year handling is working as expected. If you aren't testing that since it wasn't in the spec, than why would you have the code at all?
That isn't the purpose of all tests.
A complex public API should consist of smaller private parts. When you change those smaller parts of code, you would like to know if you break something and specifically what you broke. Testing of a small, isolated chunk of code is the 'unit' in the term 'unit tests'.
Unit tests on actual units of code allow you to more quickly isolate failures.
If the public test fails, how do you know what specific part of the public implementation was responsible for the change?
- Private methods are just an implementation detail with respect to the public methods. (You really care about whether the public methods reply correctly.)
- Public methods are just an impdet with respect to the module API. (You really care about whether the module's public surface replies correctly.)
- The module's API is JAID with respect to the user/game API. (You really just care about whether the gamer gets the right responses to controller inputs.)
- The user/game API is JAID wrt to the code/gamer interface. (You really just care about whether the gamer likes what you've written.)
- The code/gamer interface is JAID wrt the company/customer interface. (You really just care whether people still give you money.)
- The company/customer interface is JAID wrt the investor/company interface. (You really just care whether the venture throws off more money than you put in.)
Nevertheless, you'd still like to catch the failures as soon and as narrowly as possible!
Beyond that I think the key is to test the things you don't expect to change. The user always needs to be able to log in. But I expect developers will rearrange the private method boundaries.
Even for stuff that's fairly testable, like navigation, collision, etc, you have the issue of there being a million edge/degenerate cases, and it's very hard to know this never hits one in the game code or content.
So the best you can usually do is test library code.
Not that this has anything to do with AK, which sounds like plain old sloppiness.
Now, for a fast-twitch action game, processing input at 30 FPS is unacceptable; input lag sucks giant donkey balls. But I would expect that a AAA development studio knows that running input-handling, networking, physics, AI, etc in lock-step with the rendering loop is not the smartest idea. These things should run on their own schedules, independent of the graphical framerate.
This isn't a movie, it is active, not passive. "Better than traditional films" is completely irrelevant.
Take a game where you can cap the framerate and bind a key to switch between capped at 30 and capped at 60, if you're playing the game you'll immediately feel the transition and 30fps will feel relatively unplayable.
Yes, of course a frame lock sucks, but it's not the end of the world and many people wouldn't even notice if not for everyone complaining about it.
Yes, at that time (10-14 years ago), 30 fps was considered playable. But that is "playable" and even then wasn't recognised as a great framerate for most of that period.
For the last decade, less than that has been seen as bad. In fact a few years after half-life's release (when it was still popular but hardware had matured) 100fps was the goal, to match the 100Hz that most monitors back then could do. 60fps only even entered the discussion with the switch to TFT/flat panels with their lower refresh rates.
Half-life also exhibited strange behavior if the 100 fps frame cap was lifted by turning on developer mode. If I recall correctly (forgive me as this was almost a decade ago) weapon mechanics or movement speed changed.
(30Hz is somewhat common as rendering these days tends to involve a lot of fixed-cost full-screen passes for lighting and postprocessing. So by halving your frame rate you get over double the effective rendering time, which you can spend on having more stuff, higher res, etc.)
Even for 30Hz games, during development it's very common for the game not to run reliably at even 30Hz, until towards the end, at which point stuff is taken out, and assets and code optimised to make it all work. So it's not a cap that artificially limits performance so much as a floor that the developers are striving to hit :)
(I have no idea what the problem with Batman is specifically.)
I can't say with any confidence what people tend to do for PC - I don't play PC games, and I last worked on a PC game in 2006 - but supposedly one tactic I've heard is used is to interpolate object positions and properties when it comes to rendering. So you run the logic at a fixed rate (to avoid the problems I alluded to in my previous post), and save both current object states and previous object states. Then, when it comes to render, you'll be needing to render the game state as it was at some point between the most recent game state you have and the one before that - so you can imagine your latest state as a keyframe, and the previous state as a keyframe, and just interpolate between them.
I imagine this can be a bit tricky to retrofit, though, so no doubt many ports from console to PC just decide to use the same frame rate as used on the console. But I'm guessing a bit here. Unreal and the like might just do it all for you these days anyway. Maybe the Batman programmers just forgot to tick the right checkbox ;)
I'm not a game developer, but I am a realtime system developer. We could hem and haw about all the different hypothetical ways all the implementation details can suck, but locking and/or hardcoding for certain framerates is a perfectly fine decision. Doubly fine when those frames aren't graphical.
Easy but contrived example: If I hard code a purpose-specific audio pathway to only operate at 44100hz, that can be a perfectly acceptable design decision given the purpose. How deeply the code assumes that rate can be indicative of code quality, but if the assumption is hard to excise from the code in performance-critical areas, well that happens.
Here is the part where I have to qualify I'm not saying more than I've said. I'm not defending the use of 30fps cap here, or saying that this game's code is any good overall. The gaming customer has higher realtime expectations than 30fps, especially on PC. And although I haven't played this game, the market is showing it is a bad product beyond that.
I would wager that if your fix is to artificially cap performance of your software, then something in your code base sucks majorly. Call me crazy.
In a realtime system, consistent performance is a more important goal that maximum performance. For games that want high-end graphics, the goals are to have maximum performance on data of highly variable complexity, push the limits of what can be done, but rarely (if ever) whiff on the real time deadline. I won't disagree that most game codebases have pockets of major suck or that a locked rate can be a bandaid for sucking, but it is not indicative of such.
> QA/QC should not happen at the customer level.
With the exception of games running under Linux https://news.ycombinator.com/item?id=9757382 :).
A layer of abstraction above the different libraries can help, but often at the cost of a performance hit that you don't want in games.
Different input mechanisms (mouse and keyboard vs gamepad) need different input schemes to be fun.
Console games are usually played sitting on a couch in front of a TV. Sitting at a desk with a computer monitor much closer can also make a difference in what UIs (eg game HUDs) work well.
I have no clue what was involved in porting the Batman game, but your blanket statement about porting as a solved problem doesn't cut it.
An interesting example of a well-done port from PC to iPad is the game FTL. They changed the interface to feel native on the iPad. Recompiling the code was probably the least of their worries.
The X in Xbox is a leftover from its early development when it was the DirectX Box.
The X in DirectX came about as shorthand for all the various DirectWhatever APIs that Microsoft was introducing in the late 90s.
EDIT:
If the suggestion is that the porting should be relatively easy because the Xbox runs DirectX, well, kind of. It's not a 1:1 mirror of the API available on Windows, and the actual hardware is fairly divergent from what you find in PCs, even if it is x86 again.
The original Xbox was the closest to standard PC hardware, and even it diverged somewhat from what was available on standard PCs at the time at both a hardware level and API level.
The common APIs make it easier to develop cross-platform, but the platforms are still different.
So yeah, some hardware differs but it is mostly cosmetic and as a software developer it shouldn't matter to you. See https://web.archive.org/web/20080209140715/http://www.xbox-l... for more information.
I did reference the original Xbox in a later post, and I mostly had the GPU in mind there. The NV2A is close to the GeForce3 in design, but it's still not quite the same chip that was available to consumers. It was kind of a stepping stone between the GF3 and GF4 and was designed to support some things that would later show up in DX9 while not truly being a DX9 GPU.
Close, but not quite the same as standard PC hardware, and that in-between generations nature of the hardware and APIs means that optimizing for it as a target platform is going to require extra time to get it working as well on "true" DX8.1 or DX9 hardware.
http://www.eurogamer.net/articles/digitalfoundry-how-the-cre...
Not even real "desktop" OpenGL, but OpenGL ES 1.1 (branded as PSGL) implemented on top of LibGCM.
Also not to say you wrong (almost no games used any of PSGL), but at least several games used some of it. E.g Techland titles like Call of Juarez / Dead Island.
So it's GLES 1.0 + some 1.1 features + more additions.
So this is less of a port, and more of a deployment to a platform that widely varies in terms of performance. And this is much easier to do when your graphics aren't so "next gen".
Sometimes it doesn't work out, like with Batman. It lacks various shader effects like rain and looks graphically worse than on PS4. So the developers already knew the performance was a problem in their PC port and left at least two advanced effects out.
In general give the PC more love, it is more popular than one might think (international view point). PS4 is a pretty strong platform (22+ mio sold), which cannot be said from the competitors console platforms (around 10 mio or less).
There's a screenshot circulating that has 12 people listed. And I assume those aren't all developers.
But count is neither here nor there. Who knows how much was multi-platform middleware and how much custom code needed to be reimplemented.
It makes one wonder if the game is actually missing a huge chunk of files.
Alternatively, it could be "we tested it with this much free and it worked, we don't have time/budget to test again with less space."
Now Steam has introduced refunds, you can refund a game if you haven't played more than 2 hours, and a few weeks haven't passed.
(It's better than pre-order, since you get to play the unfinished game earlier, if you want to. If you choose early-access but wait for the official release before you run it, it's exactly the same as pre-order.)
Take Rust (the game, not the language) for instance. They cobbled together what is now known as "Rust Legacy" as a sort of demo of what they wanted the game to be. Thousands of people bought it. Now Facepunch is using the funds from all those sales to completely rework the game from the ground-up. Now it's better than Rust Legacy, and only going to get better and more interesting.
But if Rust Legacy wasn't fun, nobody would've bought it. TL;DR: don't preorder (including early access!) games purely on promise alone.
Once it's funded, they can build out from that base loop, and eventually you'll get a finished product.
Pre-ordering is designed to specifically work around bad reviews a.k.a. releasing poor quality games and getting away with it. The publisher is screwing you and you are paying them to.
Big publishers have no other reason to implement pre-orders. It is a shit-game risk mitigation strategy.
Pre-ordering a AAA game? No thanks. Most pre-order bonuses aren't worthwhile because they can't offer people too much or retail purchasers will get pissed off.
Limited editions are another. AKA, I WANT MY PIP BOY.
For what it's worth the game is really fun to play on a console.
Slightly off topic: am I the only one who had to open developer tools to hide the huge spoiler modal because no key combination / refresh / mouse clicking would get rid of it? That was frustrating.
My guess is, they thought even though it is buggy, they can fix it with patches and will only get minor flack for this from the PC users. And that this would be the better choice than to anger every PC gamer by announcing the third delay.
Well, now it is clear that this was the wrong decision. They both, underestimated just how broken the game was and how incomplete compared to the console version and they forgot about Valve's newly introduced refund system.
WBGames can claim that they suspended this game for PC but if I were to venture a guess, I'd say it was Valve that suspended this game because they were tired of processing so many refunds. They would have said "A game that gets this many negative reviews and all are claiming horrible performance and missing effects must clearly be broken and therefore we shouldn't continue to sell it as it will only lead to even more refund claims."
A couple of interesting things to note: 1. They removed the PC logo from all Arkham Knight pages. 2. The company itself tells people about how to request refunds 3. The company also asks for patience while this is being worked out
The last two are somewhat contradictory to me. When a company tells customers on their own free will how to get refunds sounds like an admittance to failure and that they do not believe this can be fixed in a timely manner. And then they still ask for patience which sounds like they are working on a solution. Paired with them removing all PC logos from their promo pages for this game really seems weird.
While this announcement was leagues better than the marketing speak of the first announcement they did, I think customers still need more, clearer information what is going to happen. People who deal in software already know this but the average customer has no idea how long such things can take. Therefore they should explicitly announce what they are going to do. To me, it doesn't sound like patches will do the trick. From all accounts, this pc version sounds broken by design and the root causes are stuck deeeeeep in there. So, if they plan to fix this, then they should announce that an entire rewrite is needed and that this takes time. While at it, they should also tell us who will do this rewrite. I am sure no one wants Iron Galaxy at this again. We want Rocksteady to do it. Because we know they actually can make a Batman game. As evidenced by the ps4 version of this one and the first two games in general. Both ran fine on pc. I already heard speculation that they might try and release this in the fall when they scheduled the SteamOS and Mac version of the game. One can only hope for the sake of the windows version that Rocksteady will not be busy writing these versions and continue to outsource the windows version to the same cracksquad that arsed up this time.
(In other news, I was really hoping to maybe eke by playing Knight at low/30fps on my Macbook w/discrete Nvidia graphics, but I guess that's not going to happen.)
Then, to have the absolute audacity to call a game AAA when it's hard capped at 30fps on PC is just an absolute act of spitting in our faces.
Isn't the Titan the only card with more than 6gb ram? You'd hope this would last a few years!
I'm hopeful that the coming VR PC exclusives will lift the quality of PC gaming.
Except for Rockstar Social Club...
Audio climbed this curve and has topped out. Video games haven't really explored it.
(evidently it's uninteresting enough that someone's downvoted it ..)
Very high-end systems are more niche obviously and the price onvolved their shouldn't be indicative of anything other than enthusiasm by hobbysists. Enthusiasts in any field or hobby are usually willing to spend lots of money to get a 'superior' experience.
As for waiting for higher quality I think yes, many would be willing to wait. GTA V came out nearly two years after the console releases and I was one of those who waited for it.
It's not like I was aching with anticipation for it, I had more than enough games in my backlog to play and when it did eventually release it was much better than the console versions.
It's not like GTA V is indicative of multiplatform releases though. Most will release on PC at the exact same time and not suffer the kind of issues we have seen with Arkham Knight.
Arkham Knight's absymal PC release was due only to a shitty port by a small third party studio that was obviously incapable or unequipped to handle it. I don't think I'd even really blame Iron Galaxy for this either, I blame Warner Bros and possibly even Rocksteady for thinking outsourcing the port was a good idea.
I don't have enough time to play games as much as I'd like (i.e. as much as I did as a teenager), so I only play a few selected highlights, and given that, it makes no difference to me if I'm playing the cutting edge or what came out 6 months ago - it normally takes me 6 months to finish something.
This has lots of benefits: You get the community consensus on what is worth playing, not the marketing hype. The game is usually cheaper. The major bugs have been filtered out.
Which implies, to me (both intuitively and having worked on a lot of high performance I/O limited use cases) that part of the issue is read/write speed from the disk? But if that's the case, what kind of software architecture makes sense for a game to have (apparently) real time data processing pulling resources from the disk?
Obviously you need to load maps/games/cutscene etc etc, but this makes it sound like there is constant polling every n-th frame?
[1] - http://www.pcgamer.com/batman-arkham-knights-launch-appears-...
Origins was developed console first and ported by iron galaxy. the port was atrociously bad. you could not finish the game. the main story line was bugged so hard that you could tell that the developers had never done anything more than walk around the first mission.
They had to patch to get it into a good state.
I had one game breaking bug with origins, but I found a workaround on youtube. I think origins get too much crap personally, I had fun.
This port is one of the worst in a long time. This isn't bethesda where it is just bugs on bugs, they capped the FPS.