Home Page
F-Zero Central Home Page Championships General Forum Players Photo Album
F-Zero Climax
F-Zero GP Legend
F-Zero MV
F-Zero X
F-Zero GX
F-Zero SNES
F-Zero Climax F-Zero GP Legend F-Zero MV F-Zero X F-Zero GX F-Zero SNES
Twitter Facebook YouTube Twitch
GX Rules Resources Videos Ladders Your Times Latest Times Records Forum
   « View previous topic | View next topic »  
    Index » F-Zero GX » IMPORTANT Update regarding future World Records Goto page 1, 2  Next
CGN
Staff Ghost
Posts: 315
View user's profile
Send private message
IMPORTANT Update regarding future World Records
Reply to this Post

About two weeks ago, the GameCube/Wii emulator called Dolphin got improved in how it emulates physics. To people who have been keeping up to date to some extent, Dolphin has been unable to perform identical emulation of the physics of many games, (including F-ZERO GX). This caused files transferred from Dolphin to console (and vice versa) to de-synchronize upon playback. This was convenient for world records, because this made it possible to verify whether the run was made on an unofficial emulator (like Dolphin), or an official emulator (like the GameCube or the Wii). Therefore no more than a video of the replay was needed to prove a time that had been done in real-time.

With the recent improvement in emulation, it is now possible to playback a replay file that has been made on console, in Dolphin (and vice versa). What this means is that TASes (Tool-Assisted Speedruns) can be played back on console. This is the important part, because it turns videos made from the replay mode into a useless proof for future real-time world records.

Basically, we need to come up with a proper solution to this. We don't want F-ZERO GX WRs to become dodgy. I'd say recording live from console or streaming would be an absolute requirement for future world records at this point.

What do everyone else think regarding this? We will need to change the rules for FZC, because TAS runs played back on console can be done quite easily now.

I argue that live recordings done offline are still good enough proof, because you can tell by the image quality that it was generated live from console. It would have to be in-game recordings, though, as cheating in F-ZERO SNES occured by someone who played back a TAS on his TV, with a camera pointed at it. Just in attempt to make it look like it was done in real-time on a TV. The other option would be to force a requirement of the world records being streamed and set live.

I was always a bit afraid Dolphin would become this stable in emulation, because it would lead to more strict rules. Although, in the end I knew very well that it would eventually happen. It took 11 years before the replay mode became useless for proving world records, which to be honest, is more time than I originally thought it would take before accurate emulation became a reality.

Since capturing videos live and streaming has become granted and easy nowadays, losing the replay mode for WRs shouldn't be too huge of a loss, in my opinion. People just needs to be aware and accept a change from now on. Besides, none of the other F-ZERO games have got a replay mode, and you don't see people complaining about those rules.

-

Anyways, Dolphin's improvement in emulation has got its benefits, though. First off, it should lead to TAS (.dtm) files to sync and playback more easily for future TAS videos of F-ZERO GX. Additionally, the replay mode should no longer have the potential to fail. (Not certain if the CTT TAS replay had something to do with this?).

With such a stable replay mode, it also comes in handy for real-time runs. Besides making live recordings or streaming as our new required method of proving our world records, high quality encodes can now be provided without buying an expensive capture device. We could simply just transfer our replay files to an SD card - PC, through an application on the Wii, and play it back with Dolphin's recent stable replay mode. Just take a look at how nice the image quality looks when rendered by Dolphin 4.0-2729: https://www.youtube.com/watch?v=f9yrkgB8jQc (By the way, I think YouTube's compression is better with public uploads).

The rules page for submitting WRs in F-ZERO GX will be slightly altered depending on how this topic turns out.

Special thanks to Darkeye for notifying me about this recent development in Dolphin.

Also huge props to JMC, the Dolphin developer who actually adjusted the physics to become identical to console physics. Although this topic is mostly about the concerns for real-time running, it is still great news for the Dolphin Team.

Uchiha Madao
Staff Ghost
Posts: 1971
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

well, it doesn't seem that odd to change GX's WR rules to the same as the rules used by the other games. disregarding replay mode basically puts it in the same standing as the other games where you have to record your live performance to prove a WR and stream it in case of suspicion.

also, Replay mode didn't display the actual time so it wasn't completely ideal for proof. now we have a stronger reason to make people record live for their WRs.

"Patience is useful in any moment"
RCGTiburon
Dragon Bird
Posts: 10
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

For WRs only, I think this is a good idea. I always prefer to watch a live-recorded run over replay mode anyway. I don't think requiring streaming is a necessity, and it would be onerous.

Valyssa
Wild Goose
Posts: 84
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

Personally I'm okay with this change. As Reed said, I also prefer watching live recordings over the replay mode because you just have so much more information on screen.
I'm also excited to upload some of my current PBs via a Dolphin replay for really nice quality.
Also, what about recordings starting from the result screen (displaying all times) going to the instant replay mode? Those should still be allowed right? It would probably be my preferred way of acquiring proof as recording all of my attempts is a bit impractical, though not impossible.

zewing
Golden Fox
Posts: 71
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

 I agree that all future WR's must show the live recording, which, like mentioned, isn't that hard to show.

 

Though, it's really hard to not suspect cheating when it comes to F-Zero WR's as we all know who's active and what not.  This ruling doesn't really apply to runners like me, but would you ever require video proof for non wr's?

CGN
Staff Ghost
Posts: 315
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

In case I didn't point it out properly: This new rule is only going to be applied for WRs. Although if suspicion were to occur, we might request further proof by people placing below, as well.

If you take a look at the rules page for GX (rule #6), you'll see that proof request at random has been there since the start. Some of the wording has been outdated, but it shall be updated in not too long.

yoshifan
Fire Stingray
Posts: 168
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

Well let's see. Here's what I'd have to do to use a TASed WR replay as proof:

  1. Be familiar enough with the Dolphin tools to make a good TAS run. Recording a Dolphin movie file is not needed, and screen capture during emulation is not needed. I just need to save the replay at the end.
  2. Be familiar enough with the GX mechanics to make a WR-level TAS run which doesn't have blatantly superhuman precision/reflexes. This could actually take a fair bit of effort and creativity depending on the track.
  3. Be able to run Dolphin + F-Zero GX on my computer. Dolphin doesn't have to run that well, only well enough to keep my sanity when TASing (with slowdown and savestates). And screen capture during emulation is not needed.
  4. Grab the Dolphin-saved .gci replay file and copy it to an SD card, then copy that to a GC memory card by using a Wii Homebrew app. (Maybe there is a USB Gecko thing that works for Gamecube as well, but I don't have experience with that.)
  5. Record the replay while playing it on console.

This is probably something within my ability, and I wouldn't be surprised if a few others in the community could do it too. Step 2 (GX mechanics) could genuinely be difficult though. At the very least you'd probably get people who are torn between saying it's TAS and saying it's not (although, that is already the kind of situation we probably want to avoid). But other speedrun / high score communities have had cheaters who were also very skilled and knowledgeable players, so that's not an issue I can brush aside easily.

 

Here's what I'd have to do to use a TASed "live" run as proof:

  1. Be familiar enough with the Dolphin tools to make a good TAS run. Recording a Dolphin movie file (.dtm) IS needed in this case.
  2. Know the GX mechanics very well, same as Step 2 above.
  3. Be able to run Dolphin + F-Zero GX on my computer VERY well, during video recording. This includes avoiding the various graphical issues associated with GX emulation (Raindrop mask, custom cars, needing even more CPU power for Sand Ocean's heat effect), and running at a perfect 60 FPS. If the video doesn't make framerate drops obvious, then the audio will. (Edit: Actually, the latest Dolphin's audio doesn't stutter.)
  4. Screen-capture the Dolphin movie so that the resulting video looks like an analog recording (composite, s-video, or component). As CGN said, analog recording has certain artifacts that you won't see in a Dolphin/PC recording, and you can even notice the difference after YouTube compression. But there's actually a few possible routes a cheater could take here:
  • Some older computers actually have S-Video outputs (I owned a 2006 Dell laptop that did this), so you could feed that into a capture card or DVD recorder.
  • HDMI out is much more common nowadays, so depending on whether HDMI could be passed off as component (not sure), that's another possibility.
  • There are PC to analog converter boxes as well.
  • Finally, maybe there is a way to apply filters on your PC screen capture to make it look like analog video... but that would require some technical know-how.

Step 3 might be the main sticking point here, IMO. Running GX perfectly would require more settings tweaking for sure, and an expensive computer, which is a big barrier for most people. Of course, this could change over time as either Dolphin or computers improve, but currently it's a big barrier.

Step 4 I'm not too sure about, after thinking about it more. It may only require as much money/effort as setting up a stream, which a lot of people are able to do these days... but it does take more creativity, and the methods aren't as well-known compared to setting up a stream. Also, I haven't tested any of the possible capture methods I've listed, so I can't say for sure what works and what doesn't.

 

Overall:

I think both ways of cheating would be quite difficult to pull off, though the TASed live run would clearly be even harder. I also agree that live-recorded runs are just better for proof, because they show your time and lap times, and you can't pass off a 50hz run as 60hz (this used to be an issue, right?). So I think for WRs (or anyone who suddenly comes in with insane times), requesting live-recorded runs is a reasonable thing to do.

Banning webcam runs is slightly more tricky, since using a capture card / DVD recorder could be a difficult technology barrier for some legitimate players. In fact, some current GX WRs have been recorded with a camera (see JKLoser's runs)... but game capture has gotten more accessible in recent years. And something like that F-Zero SNES case is certainly a concern; the blurriness of webcam videos makes verification really problematic. So again, for WRs, I think it's reasonable to request a proper game capture.

I don't think streaming should be necessary. That seems like too much of a technology barrier, and internet connections and streaming sites can be really unreliable. I think at a certain point you have to strike a balance: look at video quality for 95% of your judgment, and look at human factors for the rest (try to judge how trustworthy the person seems).

(A streaming requirement could make it near impossible to cheat though. I've heard of one or two cases - in speedruns of other games - where someone was suspected of streaming a TASed run, even faking reactions to the run in the process... but I think it'd be near impossible to set this up in the middle of a retry-retry-retry session in GX. It kind of works for longer speedruns because you can get a great run as your first or second run of the day.)

darkeye
Dragon Bird
Posts: 21
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

I don't really play this game any more, so feel free to ignore this, but regarding this:

Vallkyr wrote:

Also, what about recordings starting from the result screen (displaying all times) going to the instant replay mode? Those should still be allowed right? It would probably be my preferred way of acquiring proof as recording all of my attempts is a bit impractical, though not impossible.

I think this should be allowed. Since neither the instant replay nor the results screen are saved, they can't be faked using saves created in Dolphin. The race results screen is definitely stronger proof than the top 10 saved times screen, since you can fake the latter easily by setting a time in Dolphin and then transferring the save to console.

yoshifan
Fire Stingray
Posts: 168
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

Ah, missed that suggestion, but I agree with Vallkyr and darkeye. Any way of faking the results screen (followed immediately by a replay) boils down to the same methods as faking a live run, as far as I can think of.

Uchiha Madao
Staff Ghost
Posts: 1971
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

it's better to have all the safeguards in place because, while emulating gx perfectly now is pretty cumbersome, it might be much easier in 10 years and we don't want a cheater to appear by then and exploit a weak rule.

not having strong rules from the start led to some of the problems we had eventually with faked runs.

"Patience is useful in any moment"
CGN
Staff Ghost
Posts: 315
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

The suggestion pointed out by Vallkyr regarding the instant replay after the run, should be as good as a live recording, I think?

I spent some time to see if I could find graphical differences between console and Dolphin. The dash plate animation seems to last longer when rendered by Dolphin. I tested different graphic settings and restarted the emulator many times. As you can see from the images below, more sparks were made when rendered by Dolphin.

From the ten different renders I got from Dolphin, the #7 render is the render which came closest to the "common" rendering given by a console. The #1 render on the other hand, is the render which got furthest away from the "common" console render. By common, I mean that it generally generates either one or two sparks from the engine, although it ranges between 0 to 3. It's important to point out that from all of the console recordings I made, none of them went above 3 sparks. Dolphin never went below 3 sparks.

Would anyone like to test more renders on Dolphin? Or perhaps take my MCTR replay on console to see if you can get more sparks to show up on that particular frame? What this could mean is that, if there's a huge mismatch between the live console render and the Dolphin render. That mismatch would determine if it was done on console or not. This does need more testing of course, but if the difference is noticeable, we might even be able to determine if the live video was performed on console or Dolphin.

yoshifan
Fire Stingray
Posts: 168
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

Hmm, that's really interesting. The boost sparks difference is probably not something we can rely on in the long term (as Dolphin keeps improving), but it'd still be good to know for now.

If you could upload your MCTR replay somewhere, I'll do similar tests to confirm your findings about the sparks.

A useful rule for proof would ideally generalize to all instances of hitting a dash plate (different dash plates, different machines, or whatever the sparks might depend on). So it'd be something like "10 frames after hitting a dash plate, console can only show 0-3 sparks, and Dolphin shows at least 3 sparks" or similar.

CGN
Staff Ghost
Posts: 315
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

I'm still hoping that Dolphin won't ever become completely identical to console, but my hopes are probably too high.

Here's the NTSC-U file of the MCTR replay: http://www.speedyshare.com/AUnM7/8P-GFZE-fzr0000988040D3BC2BF4C393.dat-00-mctr-32780.gci

When you test, you'll need to either reset the game or re-enter the replay mode in order for the sparks to change. I tried to make a save state just before the occurrence at first, but that didn't work out.

If your results are similar to mine, we should look further into this with several boosts and different machines, as you mentioned.

darkeye
Dragon Bird
Posts: 21
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

I've got bad news for you, CGN: http://i.imgur.com/LWLod7O.jpg

In general, I don't think it's a good idea using anything probabilistic as an indicator for a "legitimate" run. Doubly so when we don't know the specifics on how it works; imagine if I was branded a cheater because my Gamecube happened to create four sparks during a replay!

In the end, banning replays as valid proof of a world record is both easier and more reliable than trying to find some strange rendering anomaly in Dolphin. Especially considering all a cheater would have to do to get around it is file a bug on the emulator and wait for it to be fixed. He could also just fix it himself; Dolphin is open source, after all.

 

 

CGN
Staff Ghost
Posts: 315
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

I suppose I must have been unfortunate during my testing. I said it needed more testing, and you pretty much confirmed that this method cannot be used as a proof.

It would have been nice if there had been something that would always be rendered incorrectly. Take a look at the emulation of F-ZERO X, and you'll see that the animation for a side-attack is still not perfect.

Thanks for doing more research, Darkeye.

yoshifan
Fire Stingray
Posts: 168
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

Yeah, I got a wide range of results with Dolphin: http://imgur.com/a/FkREx

For all of these, I used Dolphin 4.0-2826, OpenGL, and an Nvidia GPU. Between tests, I exited the replay and then re-selected the replay from the replay menu.

I also noticed that the boost would always start off with a few sparks, get a lot of sparks, and then go back down to a few sparks before disappearing altogether. But the duration of this whole pattern varied a lot. The point where it would get 0 sparks again varied by as much as 19 frames (as early as speed 2372, and as late as speed 2586). That's why at speed 2460, you'd often see no sparks at all, but sometimes there are still a lot of sparks then.

 

One thing to say about this though:

 

darkeye wrote:

In the end, banning replays as valid proof of a world record is both easier and more reliable than trying to find some strange rendering anomaly in Dolphin. Especially considering all a cheater would have to do to get around it is file a bug on the emulator and wait for it to be fixed. He could also just fix it himself; Dolphin is open source, after all.

This doesn't influence whether we should ban saved replays as WR proof or not. Even if the sparks were different between console and Dolphin, a cheater could still record their TASed replay on console, thus ensuring that their recording behaved as a console recording would. What this would influence is how well we can distinguish between *live* recordings (or *instant replay* recordings) on console vs. Dolphin. (But it seems there isn't a difference in the sparks, anyways.)

CGN
Staff Ghost
Posts: 315
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

Thank you for taking your time with Dolphin, Yoshifan.

Seems like there's not really any graphical differences that we can distinguish at this point unfortunately. I would be fine having streaming as a requirement of proving our WRs, but to implement such a strict rule right away is probably too much for the other WR competitors. For the time being, I think the best solution would be to have live recordings and instant replays (starting from the result screen with the lap splits and such) as our new requirement. To what I can tell, most are alright with it being implemented anyways. To as far as I am aware of, all of the current WR competitors have the option to at least make the instant replay going from the result screen. Besides, if you really do care about the future of F-ZERO GX, and also want yourself to be considered legit, putting an effort into it is needed.

It's not a coincidence that I've been recording live all this time. It certainly is just as much of a hassle to me, as it is to others. It doesn't really matter to me whether I recorded any WR live, or not. However, I know it is preferred by others that I do it this way, and it will only be more demanded from now on. Same goes with streaming, which I really do not enjoy doing. Basically what I'm trying to point out, is that, I put an effort into making everyone satisfied. Of course, this hasn't really been that necessary to do until now, but I hope that others will think the same way eventually, as it becomes a necessity.

I'd also like to mention something to those of you who have not been keeping up with Mario Kart Wii WRs to some extent: Quite a few people there have been doubted of their times, because they didn't put enough effort into proving their ability to play the game (by not streaming enough in this case). I really don't want F-ZERO GX to end up like so, and allowing replay mode replays alone as a proof will significantly increase the chances of it ending up like that. Live recordings / Instant replays are a much safer option at the moment. If there happened to occur suspicion from that method, we would just request further proof somehow.

CGN
Staff Ghost
Posts: 315
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

Since there hasn't been any new posts in this topic in a few days, I suppose we have come up with the most appropriate solution for future WRs?

The rules page will be slightly altered on the 15th of September then.

The replay mode alone will no longer be enough for WRs to get accepted. You'll either have to:

  1. Make a live recording.
  2. Make a recording that starts from the results screen (with the lap splits and such), and ends after the "instant replay mode" (replay mode with the replay icon shown at the bottom left of the screen).

If for whatever reason, suspicion were to occur from the above method of proving, further proof (such as streaming) could get requested.

yoshifan
Fire Stingray
Posts: 168
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

I've explained the rules proposal to the Japanese players, via Twitter: https://twitter.com/yoshifan28/status/509457127484571648

yoshifan
Fire Stingray
Posts: 168
View user's profile
Send private message
Re: IMPORTANT Update regarding future World Records
Reply to this Post

Might as well put some examples here.

Acceptable kinds of proof for WRs:

- https://www.youtube.com/watch?v=GnqisC5gvGc - Make a live recording.

- https://www.youtube.com/watch?v=E5sRMcSmLGs - Make a recording that starts from the results screen (with the lap splits and such), and ends after the "instant replay mode" (replay mode with the replay icon shown at the bottom left of the screen).

No longer accepted as proof for WRs:

https://www.youtube.com/watch?v=eXxu9HNtplg - Play a saved replay from the replay menu (no replay icon shown at the bottom left of the screen). However, old WRs like this are still valid. The rule is just for new WRs.

   Index » F-Zero GX » IMPORTANT Update regarding future World Records Goto page 1, 2  Next
 
Display posts from previous: