How much does framerate matter?

Recently we've been asking ourselves some hard questions:
  • What is it that we want to focus on?
  • What's most important to us?
  • What do we want to make?
 
And the answer is simple:
 
We want to give you guys, our fans and players, the best looking games you can buy on a console.
 
You may have already seen Ratchet and Clank Future: A Crack in Time (available in US stores now!)
 
 
I'm really proud of what our art and production teams accomplished in this game. It's a great looking game, a ton of fun to play and is 60fps.
 
And it's that last point that I want to talk about today. One of the long-standing sacred cows here at Insomniac is framerate. We’ve long viewed a solid framerate as both a sign of a quality product and professionalism as developers. It’s always been point of pride in our work and considered an extremely serious part of our development process.
 
However, during development, there are hard choices to be made between higher quality graphics and framerate. And we want to make the right choices that reflect our commitment to providing you with the best looking games out there. To that end, our community team did some research into the question of framerate. The results perhaps confirmed what I’ve known for a long time, but found it difficult to accept without evidence. They found that:
 
  • A higher framerate does not significantly affect sales of a game.
  • A higher framerate does not significantly affect the reviews of a game.
 
And in particular they found that there was a clear correlation between graphics scores in reviews (where they are provided) and the final scores. And they found no such correlation between framerate and the graphics scores nor the final scores. As an interesting side-note, our team also found no direct correlation between gameplay scores and final scores, however it does appear that gameplay scores are also influenced by graphics scores. i.e. Better looking games appear to be more “fun” to reviewers, in general.
 
 
 
After reviewing our internal research, I decided to take this question to the public. I wanted to see what the players themselves thought of this question. Here are the results of that poll
 
 
The first thing I noted in reviewing these results was that 16% of the respondents said they wouldn’t buy a non 60fps game. Now, considering the top selling games and the market research, I take that to mean one of two things:
 
  • People are big fat liars. Sales numbers clearly contradict this pattern. Or,
  • The group responding to this poll in the first place was a self-selected group of people with an interest in framerate in the first place. Which may also explain why that last group is represented by such a small response rate in the poll results.
 
Based on the research, the informal polling and various conversations with fans and other game buyers, I’ve come to the following conclusions: 
 
  • Framerate is important, but not critically so. When there is a clear choice between framerate and improved graphics, graphics should win. The correlation with review scores is clear.
     
  • There is virtually no advantage in sales or reviews of a 60 fps game versus a 30 fps game.
     
  • Only a minority of players notice framerate as a significant issue of any kind.
     
  • Framerate should be as consistent as possible and should never interfere with the game. However, a drop in framerate is interestingly seen by some players as a reward for creating or forcing a complex setup in which a lot of things must happen on the screen at once. As in, “Damn! Did you see that? That was crazy!”
     
  • A solid framerate is still a sign of professional, well-made product. When there is a trade-off for framerate, it needs to be clearly worth it. i.e. It must introduce clear improvements on what the player sees, and never used as an excuse to not optimize the game or art.
 
 
What does all of this mean, really?
  
It means that framerate is still important to us here at Insomniac, but it’s not on the same pedestal it was before. And that Ratchet and Clank Future: A Crack in Time will probably be Insomniac’s last 60fps game.
 
 
Mike.
 
PS (Update 30 October 2009)
 
 
dixee wrote:
Plus, there are other advantages. You can load things twice as fast (assuming it's being streamed during gameplay) and calculate twice as many collisions. Frame rate involves more than just drawing methods, unless I'm drastically mistaken. Am I?
 
You really can't load things any faster regardless of your framerate. The speed (seek and throughput rate) of loading is limited by device you're loading from (e.g. the Bluray). But yes, a lower framerate does give you "extra" time to do other operations as well, outside of rendering. Collision, as you mentioned, is an example.
 
Rhez Darkhoof wrote:
Framerate is nice but gameplay and graphics probably have more effect on the overall experience. I mean how high can we really get the framerate before the human eye can't distinguish the difference.?
 
The human eye and brain are seriously sophisticated equipment. It's certainly possible to distinguish very high framerates. Much higher, in fact, than 60fps. But the question is more about how important is it to most players (and reviewers) when when offered the choice against "better graphics" of some kind. We want to give players better graphics, because ultimately that's what so many are asking for.
 
Arelius wrote:
Mike, I'd like to make a few strong counter points. First of all, regarding your Poll, and the fact that 16% said they wouldn't buy a non 60 FPS game. Keep in mind there were no other "I prefer a 60 FPS game" option, so while the exact text may have said "60 FPS or I won't buy" it got people saying "I prefer 60 FPS and it influences my purchases."
 
So, yeah. I should have clarified the point. That poll was extremely informal and didn't play a significant part in the decision process. It was more a point of interest. However, we can see through sales data what people are saying when they speak with their hard-earned money. And we simply don't see any significant correlation between framerate and what people want to buy.
 
But I don't want to dismiss the importance of framerate, either. As I said, it is still important to us. We certainly understand how framerate can affect the game experience. And we definitely have people here that prefer 60fps as well. But when it comes right down to it, when we have to make a choice, that choice has to be made based on what we think you guys will like better. And what you'd actually prefer to buy, in general. It's not a perfect system.
 
 
mgibson wrote:
That graph of overall scores and graphics scores makes no sense. The horizontal axis is meaningless.
 
Just to be clear, what I wrote was definitely not about the details of our research. There is clearly a lot of information that I didn't present. Nor did I present it in a way that could allow for any kind of rigorous analysis. I'm more speaking about what conclusions we've drawn and why we think it's interesting. 
 
The horizontal axis represents the individual games. The vertices are the only things that have any meaning. The lines drawn between them are not meaningful data themselves, but do help visualize the pattern for an informal view.
 
MiZa wrote:
I know, my words are wasted though as soon as I read your mission statement. If it reads "We want to give you guys, our fans and players, the best looking games you can buy on a console" instead of "We want to give you guys, our fans and players, the best games you can buy on a console" then there is no hope.

That's a good point, MiZa. And we do want to provide you the best games you can buy on a console. Though here, I'm speaking specifically about engine (and art) choices, and ultimately this question is fairly narrowly focused on the A/V experience.
 
superdynamite wrote:
Let me start off by saying that frame rate plays a tremendous part in the gameplay of a "Next-Gen" videogame. The statement that was made by Insomniac, sighting "no correlation between 60 fps benchmark to yield higher retail sales" is complete and utter nonsense,
 
I appreciate your view here, superdynamite. But I'm going to stick by our research. Certainly you can look at many current games that run at 30 fps and ask if their sales have been impacted by not being 60 fps. Additionally, I'd be surprised if you could find a single review (in similar genres of games that we work on) that penalized a game for not being 60 fps.
 
Thanks so much for all of your heartfelt feedback everyone! And keep in mind that I'm certainly not saying 60 fps has no value or that framerate isn't important. It's simply about focusing on what we think is most important and devoting resources to that. So that when you pick up our games, you know you're going to get an awesome graphical experience. Along with great fun.
 
 
 
 
  • awesome_farts

    Frame rate over graphics!
    Smooth motion is key.
    Not impressed by this design choice. But atleast graphics improved.

  • http://www.iamthebrandmasrer.com RockyVega

    I beg to differ. Frame rate can have a lot to do with the visual experience of the game. Depending on the look you’re going for, you can wind up achieving the exact opposite of what you want with the wrong choice of frame rate.

    Frame rate shouldn’t be looked only for the purpose of performance but, also for the sake of overall design. A “movie-like” experience, for example, will be better achieved in 30 fps while a 60 fps will give more of the impression you’re actually doing something in real-time.

    It’s the same difference you get when you watch a movie versus a TV show. Your eyes aren’t deceiving you. The movie actually looks different than the TV show. It’s because of frame rate. TV Shows tend to be shot at 29.97 fps or higher while the standard for movies is 24 fps.

    What tends to look more professional to people? Movies.

    So, again…frame rate is important. But, I would never go so far to put it over the visuals every single time. I’ve played plenty games myself at 30 fps that may have not had the same feel were it at 60 fps instead and vice versa.

  • UMAD BRAH

    This makes it sound like money is way more important to you than the games themselves. The masses aren’t that picky about framerate because they simply don’t know about it. I’ve been a fan of the R&C games since the start and a lot of that had to do with how smooth the gameplay was. Framerate is more important than graphics and into the nexus is a fine example of that. I couldn’t stand that game for more than 20 minutes because the gameplay felt so bad coming from the older R&C games. Probably won’t be picking up the PS4 R&C after all.

  • Justin (~≧▽≦)~

    The moment they realise their highest rated game ever was a 60fps game…

  • Brandon Collins

    The only reason you guys have to even deal with this debate is because current console hardware is just not up to par with the technology available. The Xbox One and PlayStation 4 have to make sacrifices between graphics and frame rates because they’re underpowered. In a vast, growing graphical industry with every game developer trying to keep up, it’s just not plausible to have both aspects in a $500 machine. And especially when you develop a game for platforms that don’t allow graphical customization (so being able to change shadow quality, shading quality, texture quality, etc. in order to sacrifice graphics for frame rates and vice versa), no one’s going to be happy. A good PC basically fixes all of those problems, but even then, if we’re to be honest, a PC for the same price IS a better performer, but it’s not so much better that it will completely fix the problem. A PC for around $700+ is where games won’t have to make that compromise to the extent it’s at now with current consoles. Basically every PC game has customizable graphics and I don’t know why console games have never usually never implemented it. No, they lock it at a certain graphical quality and frame rate and expect everyone to just be okay with it. It just doesn’t work like that anymore because of how demanding games are getting and how much more gamers are preferring one choice or the other. Maybe if you guys started implementing graphical customization in your games, this wouldn’t be a problem and gamers could make their OWN choices. Or, you know, ditch the PotatoStation and start developing for PC…

    • David Romao

      Ditching the potatoStation isn’t possible because R&C belong to sony.. unfortunately
      I can’t imagine how good would be a ratchet game made for PC

      • Brandon Collins

        It would be Ratchet and Clank on a PC? I mean, you can hook up a PS2/PS3 controller to the PC if the oh-so-glorious mouse and keyboard don’t suit your R&C needs. 😛

        But yeah, I kind of figured the franchise belonged to Sony. How else will they get developers to create games for their potato other than buying out entire franchises, alienating other platforms?

    • UMAD BRAH

      I think the consoles should have a lower graphics 60FPS / higher graphics 30FPS option like you mention. There’s no reason why it shouldn’t be possible, the consoles are pretty much budget gaming PCs. Many people jumped over to PC gaming between last console gen and this console gen and it’s really hard to go back to 30FPS when you’re used to 60 or higher.

  • Jeremy Venegas

    “We’ve long viewed a solid framerate as both a sign of a quality product and professionalism as developers… And that Ratchet and Clank Future: A Crack in Time willprobably be Insomniac’s last 60fps game.”

    wut

  • http://www.flickr.com/stuffbox DuduMaroja

    a great gameplay allways comes first.. with rock solid 60fps games are not only plays better but feels better..

    and no.. whenever i see a framedrop.. i think… “rushed f… game”

  • UMAD BRAH

    Just because framerate isn’t mentioned in reviews doesn’t mean that it doesn’t matter. Stop making excuses. Platformers do not play good at 30fps.
    There’s a good reason why PC multiplats (if the review is fair) gets a higher score.

    • Soushi

      Also worth mentioning, that console gamers don’t have a choice in this matter: they are forced to play the game with framerate, that was decided by developer, without any options to raise it.

      If there were a game, that was released with 60 FPS on one console and 30 on another you’d see a noticeable difference in sales/scores between the two (assuming most gamers have both consoles and assuming the press would review both versions back to back).

  • CJ Pony

    One final screw you for ruining such a magnificent game with a shitty sub-par framerate.