| Mail |
You might also like: WoW Insider, Joystiq, and more

Reader Comments (17)

Posted: Nov 30th 2009 3:48PM Pingles said

  • 2 hearts
  • Report
I am pretty much only playing MMO Betas nowadays.

A couple of the developers hinted at something I think very few Beta-Testers seem to consider: The developers focus in a Beta-test are not likely the problems that the testers are reporting.

They are looking at raw server/game data and monitoring CPU usage, memory, etc. and their focus may be on things that players will never see.

When servers crash during launch it's because developers haven't done their job cleaning up the servers software.

There's always a handful of idiots in every Beta forum posting stuff like OMG YOU CAN C UNDER THAT ROCK LOL DUN U CARE ABOUT UR GAME LOL I POSTED ABOUT IT 2 DAYS AGO STILL NOT FIXED LOL

I am sure that the forums get screened and problems cataloged but a lot of those tweaks can be made late in beta or even just before game goes live.

Posted: Nov 30th 2009 4:06PM Anatidae said

  • 2 hearts
  • Report
Ugh. Betas have become marketing schemes more than testing platforms these days.

Plus, there is always the danger of the developer not listening to their customers. Heck, even Walmart listens to their customers - they might treat their employees and vendors like crap, but they will wipe your butt if they thought it would lead to you buying more goods from their store.

With betas becoming more PR stunts, there seems to be a further disconnect between customer and developer. Communication gets, er, funky.

All that aside, we have had a year of MMOs that are devoid of depth. Some with great ideas (WAR) but lacking in a certain spirit. I would contest that the MMO genre needs a shot in the arm of inspiration.

However, I have hope. I am guessing ArenaNet will have something fresh for us in GW2. Games like Earthrise and other sandbox MMOs might truly bring the next generation Ultima Online experience where the world feels alive - if they can implement tools for the customers to counteract the negatives of a free-open-PvP experience.

Ironicly, I would be willing to bet that the next leap in MMO evolution that truly rocks will be from Blizzard. Only because they are not going to muddy their own income with a clone of WoW. They will think of something that is evolved enough to set itself apart while still allowing WoW to do what it does best. Then everyone will have a new model to copy.

Posted: Nov 30th 2009 4:12PM Russell Clarke said

  • 2 hearts
  • Report
Looks to me like apart from the stress testing aspect, the betas are actually more like UAT (user acceptance testing).

Posted: Nov 30th 2009 7:45PM (Unverified) said

  • 2 hearts
  • Report
You dont enter into a NDA when you join an open beta, just a closed beta.
Reply

Posted: Nov 30th 2009 4:23PM CCon99 said

  • 3 hearts
  • Report
Yeah the Cryptic answers were really funny. Bill Roper mentioned in an interview just last week how CO's launch was really hurt because they didn't properly Beta Test and didn't have enough time to collect the proper data.

So knowing all of this, what do they go and do with STO? They go to the same 4 hours per week testing schedule that they already admit failed for CO. At least CO had 10 months in beta, STO's only getting 4 months if there is no delay.

Posted: Nov 30th 2009 4:28PM (Unverified) said

  • Half a heart
  • Report
How dare a company actually try to make money.

Don't be an idiot.

You're probably the kind of loudmouth crybaby that all of the guys interviewed warned against catering to.

Posted: Nov 30th 2009 4:35PM Holgranth said

  • 2.5 hearts
  • Report
I agreed with everyone except Emmert, to some degree or another a few things I disagree on with all of them.

Jack Emmert, holy cow I'm never touching anything he has.

Single best quote goes to Peter Hu:

"Being defensive about the game. The purpose of the beta is to make things better. It isn't about coddling the developer's ego."

Posted: Nov 30th 2009 4:45PM breezer said

  • 2 hearts
  • Report
Funny, that was THE standout line of the article for me too.

To give him the benefit of the doubt, he could just be saying that it's ideal to make changes that will improve the product, thus making the product more appealing to potential subscribers.

What it SOUNDS like though, is that it's ideal to add superficial changes that dress up, or add gimmicks or selling points to scam suckers into buying the half-assed crap they keep pumping out.

Posted: Nov 30th 2009 4:58PM Minofan said

  • 2 hearts
  • Report
An interesting read to be sure - thanks for doing the rounds, Massively. ^_^

Got to say I noticed a common absence in the recurring theme of "all those greedy players are expecting the finished product early for free!" sentiment - which is of course true; no mention of how developers (including some of those interviewed) are going beyond marketing and blatantly SELLING their beta as a finished product.

* Cryptic with their life-time subscription offers expiring before launch.
* Cryptic again (though certainly others in various forms) selling beta access with other purchases.
* Turbine selling their non-refundable item-mall currency fairly aggressively throughout beta.

I cannot deny that beta-as-UAT may be a dangerous development practice, but its hard to feel sympathy for developers when some are diving headlong into dangerous business practices that are - to be frank - in danger of tarnishing industry cornerstones irreparably (certainly I'm passed the point of entertaining offers of lifetime subscriptions or collector editions from quite a few companies now).

Posted: Nov 30th 2009 5:18PM GreenArmadillo said

  • 2 hearts
  • Report
Almost all of the major MMORPG launches of the last three years have offered beta access as a pre-order bonus. This means that players gave up cash (albeit in the form of a sometimes-refundable pre-order deposit) in exchange for a promised service (access to the beta). It's no wonder that beta populations are looking more and more like customer populations with customer expectations, when they are, in fact, full of customers.

Posted: Nov 30th 2009 5:41PM Graill440 said

  • 2 hearts
  • Report
The devs read as clueless as the ones programming from their basements. Since the industry is not held to any standards we have what we have, a mess of lousy MMO's and devs stating for the record that peoples "standards have risen in the last two years..........."

If that single statement doesnt make folks cringe at the thinking, ignorance and ineptitude of devs in the last 15 or so years to present i dont know what will. Sad.

Posted: Nov 30th 2009 6:59PM Averice said

  • 2 hearts
  • Report
I've got to disagree with your opening statement. Many developers have used their betas to showcase the game and pretend as if they're going to make it better, when in fact they do not ever actually make the game better. This is the reason why so many players view betas as the final product and a free trial, instead of what it was originally, a beta.

It's really down to the reputation of the company these days, more than just passing everybody off as looking for a free trial. I remember many betas I myself have applied for around a decade ago, where the first thing they ask is "have you participated in any betas before? And please give examples of what kind of feedback you provided." Well, nobody does that anymore, it's always open beta, or closed beta for preorders. Those kinds of betas aren't real betas, they are merely showcases, and again it is not the consumers who have designated it as such, but the producers.

When you use beta keys as a promotional tool instead of a serious process, the only feedback a company is looking for is an outlook on how consumers are going to feel about their product. I think there's a disconnect here between what a private closed beta is, and a public closed beta, they are very different.

Posted: Dec 1st 2009 11:06AM (Unverified) said

  • 2 hearts
  • Report
Averice: "When you use beta keys as a promotional tool instead of a serious process, the only feedback a company is looking for is an outlook on how consumers are going to feel about their product. I think there's a disconnect here between what a private closed beta is, and a public closed beta, they are very different."

Agree 100%, as well as with the comments here about the Cryptic rep. His "theme" of the players being at fault seems especially ridiculous right now considering they are "delaying" fans of STO entrance into beta while they make way for random contest winners.

Note to developers: If you want "professional" style testers, then use "professional" methods for selecting testers. Beta keys to the first 1000 posters on www.mmosareleet.com is NOT the way to do it.

Perhaps the biggest thing I noticed in the interviews was the movement away from "closed" betas. Heck, I don't even think that all four participants could come up with the same "definition" for a closed beta!!!!

That trend is what, IMO, has been the biggest development in the last few years. It once was the norm to invite players into the game nearly a year before release to help preparing the game. And, as someone pointed out earlier, one used to have to be an active member of the games community, with some testing experience, certain computer stats in order to test, and were required to sign a strict NDA. "Closed" betas were definitely NOT free trials of the game...they were WORK.

Maybe players just didn't want to do them anymore. Maybe developers can't take the time to work the "closed" beta test due to the money constraints of modern mmo development. Or maybe devs just don't want players interfering with their "babies" that much. Whatever the reason, I think THAT change is KILLING the development process.

The fact that core fanbases, found in forums and on fan-created sites, are not being utilized, and instead, public playtests are not the ONLY "definition" of beta(and companies like Cryptic truly believe they "can't do anything about it), will KILL this industry. Mark my words.
Reply

Posted: Nov 30th 2009 9:05PM Dumac said

  • 2 hearts
  • Report
Well frankly what else is a beta for but a free preview? Do you get any tangible rewards from such an endeavor? It is probably 'painful', with crashes, bad client and server performance, lots of bugs etc, surely you have to have a motivation to go through such pain? Yet the studios do not really reward your time and nerves, you never get a free account when the game goes live, you never get some extra free time on top of your free month when you eventually buy it, you never get a special limited to beta testers sword or stuff...

All you get is a "special thanks to the lovely beta community" in the credits and that's all. Of course, the only thing you really get from it is an early preview. Say its bad all you want and that might even be true, but i do not see any other possible motivation for beta testing.

Posted: Nov 30th 2009 9:56PM Lionhearted said

  • 2 hearts
  • Report
You don't sound like a good candidate for a real beta, which is fine, because most people aren't. Some people are interested in beta because they're just plain, old interested in games -- not the actual games themselves, but the design and making sure things work, the mechanics in the game which make it fun or make it suck or something in between. This aspect of games has actually been far more interesting to myself than the questing, the hunting, etc. etc. etc.

Furthermore, some people like to know that they had a stamp on the game. Playing in a beta for someone who feels that way is probably something akin to someone who wants to be a stand-in at a movie, somewhere in the background. They rarely ever get paid and all it is is trouble for them, but they feel as though the movie they were in --- for a split second --- is somehow a part of them. Several movies have been filmed in my town over the past few years and I've had friends who stood by for hours to get into some background shots. LOL.

Gamers, on the other hand, may want their stamp on the game, their ability to say 'i was a part of getting this game off the ground,' their story, etc.

That's not for everyone. I'd never stand at a movie location for hours trying to get in. On the other hand, I love games and getting to have an impact on making them better is something I *would* be willing to spend some time on (I have done some real betas in the past, back when betas were really done..). Each to their own.
Reply

Posted: Nov 30th 2009 10:01PM Lionhearted said

  • 2 hearts
  • Report
I have to admit, the guy from Cryptic's statements about the beta made me much, much more afraid about STO.... which is a shame, because I've been dying to play it. I was very confident in this game a few weeks back, but I'm starting to worry about it -- and this Cryptic Rep did nothing to improve those worries.

Posted: Dec 3rd 2009 9:20AM UnSub said

  • 2 hearts
  • Report
Having read through all the developer comments, I'm amazed that people singled out Emmert for his comments on the player base given that the same kind of comments were repeated by other devs. Perhaps he's most recognisable with ChampO out not so long ago and STO coming, but apart from mentioning money, he's really only echoing the same kind of comments.

The reality is in all of these dev comments is that no studio really follows them. They might want to - as an example, FE's dev spoke about communication with the player base but imo they were awful at it because the vast majority of comments came from CMs, not devs, and contained no real info - but the reality of business is that it doesn't happen. Not every player gets listened to. Not every idea from the tester base, no matter how good, gets implemented. What one person considers good communication is perceived by another as being ignored.

Ultimately, that's the difference between intent and reality - I'm sure devs would like to have open testing until they get it right, but all that costs money and players don't hang around to test incomplete systems because there are plenty of polished games out there for them to play.

Featured Stories

Engadget

Engadget

Joystiq

Joystiq

WoW Insider

WoW

TUAW

TUAW