| Mail |
You might also like: WoW Insider, Joystiq, and more

Reader Comments (20)

Posted: Aug 6th 2009 8:17AM Daverator said

  • 2 hearts
  • Report
It is both.

The closed beta period allows them to have thousands of testers for free. Now these testers aren't half as dedicated as a paid QA tester, heck maybe not even 1/10 when it comes to reporting issues. BUT it turns out they are a heck of a lot cheaper.

The open beta period however, usually serves to stress test servers, to get a handle on how many people a server can actually handle. And also it serves as a demo of the game for the masses. Two weeks before release means nothing major is going to change. Sure if something horrible is found it will be fixed, but not likely in a week.

That last part is what irks me about Champions Pay to Play "Open" beta. You are asking people to put down $50 to rent your game for 1 month, (and the privilege of renting it for subsequent months for only $15). Thats a hefty rental fee for something that very well may be played for a short period and discarded, so disallowing people to even try it in the most limited fashion before committing is a bit off putting.

Posted: Aug 6th 2009 8:28AM (Unverified) said

  • 2 hearts
  • Report
A beta is (supposed to be) an even exchange.

Players are asked to donate some of their time to help test a new product, report any bugs they find, and participate in feedback surveys. This information is valuable to companies since they can't even hope to employ enough professional testers to explore every inch of an MMO, run realistic load tests, monkey around with game balance scenarios, find loopholes, etc.

In exchange for their time, players receive a sneak peek at a product that they are excited about, and sometimes receive in-game perks. This experience is valuable and co vetted by (most) players who love new shiny things and the status that being a beta tester can sometimes bring within a community.

Unfortunately it seems that most companies have realized that fanboys are more than willing to pay money to perform this valuable service for them, and so you see pay-to-play betas, or pre-order only betas, or insane "deals" like the recent "buy a lifetime sub to CO and get into the STO beta:" ripoff.

Posted: Aug 6th 2009 9:24AM Macabre 13 said

  • 2 hearts
  • Report
This!

These days, it feels like as if we went to a restaurant and they said "order this steak and we'll let you cook it yourself!" Joy...

A part of me even doubts how much they really even value and/or utilize the "beta testing" these days, and instead just use the traditional system as a hype machine, as it seems like there's just less and less difference between beta and launch versions...

I don't think casuals or randoms are generally beta testing. Those in a beta are generally already fans in one way or another, and probably have been following a game for months (even years), and as such have been drawn to that specific game for whatever reason. So there's a bias, and this seems to hold true, as I generally don't read too much negativity from beta testers, even from those games that flopped when they went live.

Who better to hype your game than fanbois, no?

But even if the above holds true, I'm still perfectly fine with that, as long as the traditional exchange remains. When they start charging an admission fee, then it starts to get irritating.
Reply

Posted: Aug 6th 2009 8:31AM FrostPaw said

  • 2 hearts
  • Report
Yes it is free labour, but its also a free game so people are willing to do it.

On the other hand, its free labour that rarely does what you want and often is particuarily innaccurate when it does. Reading some beta forums and complaints its easy to see how crap some testers are at giving feedback or reporting bugs.

I think the developers weigh up the pros and cons, if only one in every hundred testers gives them usefull information thats still usefull information they didnt pay for.

Given how widely varied testers systems are the potential for finding something that would affect an important part of your customer base is worth the aggrivation of having hundreds of free loaders.

I'm not convinced the volunteer testers get the kind of experience they hope for when they apply but even if they decide not to keep testing there are still plenty of others queueing to take their place.

Posted: Aug 7th 2009 5:13AM (Unverified) said

  • 2 hearts
  • Report
Both.
You're doing something you enjoy.
People usually have to pay to do something they enjoy.
Now you do it for free.

Makes you wonder if there is a way to make labour fun enough to make everthing free.
Impossible? Yes, in this day and age it is very impossible.
But not always. There are alot of activities that create a profit for SOMEONE, and is being done by people who enjoy it and do it for free.

Posted: Aug 6th 2009 8:45AM (Unverified) said

  • 2 hearts
  • Report
I do beta testing for 2 main reasons

1. To find out if the game is worth purchasing. As like thain says it usually costs us a fairly large sum for us to "buy" it and on top of that theres a monthly fee to consider. Before I fork out my hard earned cash I'd like to know if I would enjoy it for more than a week.

2. To find out if my system can run it. My system was mid range 3 year ago and is now showing its age. Recently Ive tried games with min systems on the back that are below that of my machine and I find I have 2 choices, I either have a game that looks half okay that is unplayable or I have a playable game that looks like roadkill.

As you cannot exchange sell or get refunds on online games a little advance knowledge before buying is invaluable.

Posted: Aug 6th 2009 8:57AM (Unverified) said

  • 2 hearts
  • Report
I really don't understand your point.

People play betas because they want to play betas. People write bug reports because it makes them feel good to have contributed. There is nothing shady or unethical about beta testing. It is not 'free labour'. It is simply early feedback from a wider audience. Developers benefit from feedback, and people are given the opportunity to take part if they wish to.

"Are betas just a cheap ploy to get a few thousand fresh pairs of eyes or a genuine attempt to gauge a new game?"

Those two propositions are identical, once you take out your arbitrary and ridiculous 'cheap ploy' remark.

Perhaps you should have waited until after your morning coffee before posting this up.

Posted: Aug 6th 2009 9:37AM Macabre 13 said

  • 2 hearts
  • Report
I completely agree, I don't see it as a cheap ploy, and I also don't see it as free labor.

I've been in plenty of betas, and I never felt like I was "working", even though I try to contribute as much as I can.

A community beta tester and a paid QA tester are nowhere near the same thing. The requirements are worlds apart (i.e., there virtually are no *requirements* of beta testers, simply suggestions to provide feedback). I've heard and read stories and job descriptions from QA guys, and I do not envy them one bit, as I couldn't handle what they do. Any actual QA testers care to chime in here?

As I said in a previous reply, the exchange of a beta test is perfectly fine, but the new concept of "admission fees" are definitely off-putting.

Posted: Aug 6th 2009 9:40AM (Unverified) said

  • 2 hearts
  • Report
I'm guessing CO is using Open Beta Stress Testing. CO is probably going to use the next few weeks to fine tune their networking code and account signup capacity. Unless something is horribly broken, any new bugs will be checked against whatever internal list they have first.

Posted: Aug 6th 2009 9:49AM (Unverified) said

  • 2 hearts
  • Report
Hmmm... Sorry Lesley but you need to approach the question better if you want it to be thought provoking.

I don't think beta testers fall into just those two camps. That's pretty silly to assume. Not only that, but I think often times the cost of holding widespread betas ends up being greater than the benefit.

I know there are NDAs and I know there are rules to keeping things tight lipped, but in my experience betas are review time. Whether people put out their reviews or hold it in until the NDA lifts... it's time to analyze and critique an unfinished product.

Listen, I'm a graphic designer. Often when people see my work before it's done they're like "really?" It's super frustrating because I have to explain to them that they don't see what's going to eventually come out of it. However, once people make a decision on what they think about something, they sit on it until you go the extra mile to prove them wrong. I mean the extra 10 miles.

For example, when I show people early work to see what they think, they take it as "oh, you want me to criticize it.." rather than also consider complimenting it. They throw out all manner of crappy suggestions (there's a reason I'm the professional artist and they're not) and then decide "well it's not what I would do, so its not good". On the flipside, if I don't show them work until it's done and awesome and say something like "hey, look what I did", they much more often than not love it. Not only that, but now they trust me so I can show them concept work and they have a tendency to trust that it will end up great despite it not being done.

Point is, I don't think betas serve all that much good and I don't think companies are leeching anything from consumers. I think they're putting themselves on the chopping block way too early and getting people decided on a game well before it's even done.

One small disclaimer, I'm not saying that everything is all one way all of the time. I'm just stating that I think betas serve gamers and their need to play MOAR GAMES and hype/buzz more than anything. Other wise, when weighting the cost/benefit of beta help versus early critique, I think it leans more heavily as a negative.

Posted: Aug 6th 2009 10:05AM (Unverified) said

  • 2 hearts
  • Report
No. For every hard-working tester that submits detailed bug reports there are a dozen people who just see it as a free demo, so both sides seem to benefit from betas.

Posted: Aug 6th 2009 11:13AM Anatidae said

  • 2 hearts
  • Report
These days betas are a marketing ploy to get people interested in the game. For example, Champions Online.

They had contests all over the place for the last month giving away thousands of "closed beta" spots. From what has been posted on a couple forums and talked about in the IRC channel, the testers waited weeks for their keys only to play for 6 or 8 hours and now they are locked out again.

With Open Beta only a week away really, this is a good example on where the PR team of Cryptic used the lure of "closed beta" to create hype. Sure, they delivered, but not really that much more special than the other people who will slide in with other giveaways this week in anticipation for the open beta.

What I am saying, none of this about the testing. Sure, the beta players help the developers, but when you give away 10,000+ keys, then make people wait a week or two for them, then make them wait almost another week to then open the server for testing during business hours... well, that is not a big window for testing. Hence, it is all a PR stunt.

In fact, I wonder what Massively gives to Cryptic for the beta key giveaway on this site? Is it just to create more articles here? Or does Cryptic also get a discount on ad banner space?

Posted: Aug 6th 2009 11:19AM Cendres said

  • 2 hearts
  • Report
I see a lot of people say we now play to beta test, but these companies still invite a number of people without any pre orders at all. If you are really interested in a game and following closely usually they are a number of ways that become available that gives you a chance to get in the beta, closed or open or whatever.

The pre order priviledge of being admitted to beta to me is just another way to market the game, I'm not sure if it helps the game or not but it's a nice way to hook some people in and in this business I can't say I blame them.

Personally I do see a beta as a way to preview a game but at the same time I do my best to report anything I come across that's not right while I'm there. It's not work it's just fun :)

Posted: Aug 6th 2009 11:26AM charlieromeobravo said

  • 2 hearts
  • Report
"For these folks, betas (while fun and exciting) are just an excuse for Company X to save some money by getting a few million free and eager quality assurance monkeys. "

And they're worth ever penny they get paid.

The first small entrants into a beta program are probably useful. They're usually the hardcore players who are very familiar with the mechanics of MMO game play. After that though? Open betas? My feeling is that they're more marketing initiatives than tests. When the beta goes open, they just attract curious players, not people who are genuinely testing anything in any concentrated and organized fashion which would be truly valuable to the developers.

Posted: Aug 6th 2009 12:03PM (Unverified) said

  • 2 hearts
  • Report
Uh, I'd like to point out that Quality Assurance "monkeys" do not get paid minimum wage. I found that a tad condescending, especially considering the audience of this article.

Posted: Aug 6th 2009 12:12PM charlieromeobravo said

  • 2 hearts
  • Report
This is true. Good professional QA people are hard to come by and get paid decently.
Reply

Posted: Aug 6th 2009 12:47PM (Unverified) said

  • 2 hearts
  • Report
The strength of a company's QA department can make or break it. As a professional developer, I value our test team's work extremely highly. They are every bit as professional as the development team itself.

Also, as someone who has had to perform QA from time to time, it's neither trivial work nor simplistic work.

Professional testers need to have strong comprehension skills (to know what an app is supposed to do), excellent observation skills, an analytic mind (to create a reproduction scenario and/or speculate on the source of the problem), and a solid grip on technical writing.

Calling QA departments a bunch of "monkeys" is an easy way to reveal your own ignorance.
Reply

Posted: Aug 6th 2009 2:31PM NeoWolfen said

  • 2 hearts
  • Report
I don't really think its quite that simple myself..we arent all either one camp or the other.

Although some may undoubtedly fall exclusively into one camp or the other there are others like myself who consider themselves (at least partly) to belong to both.

I have playtested for quite a few years now.. firstly with PnP products, later with computer games and then finally with MMO's... for me it has always served three causes.

Firstly, I only apply to beta test games I have an interest in, those I have followed the development of, and am curious enough to want to play. So beta-testing in this respect gives me a chance to preview said games prior to release in the same way a demo for an offline game works.. however as the games are in BETA.. it is always to remember not to formulate too harsh an opinion of a game being beta tested as it is afterall in a period of development where their will be things wrong, inbalances, plain broken mechanics and so forth.

Secondly, I also see testing as a means to ensure those game I DO wish to play end up being released in a state that is such that I can and will enjoy them when they actually do go gold. So yes I am one of those testers who DOES actually test the game, try to break it, does actually look for inbalances, broken aspects, suggests changes/improvements etc..

Testing is also great for a third much overlooked aspect, notably that of guaging not only a companies business approach but also thier seriousness and responsibility toward thier game and thier customers. Do they listen to us? If something in the game is broken do they care enough to fix it? How quick are they to respond to us or a problem etc... It gives you a great means to see how serious they are about the game they are making and whether or not they are somoene you would feel comfortable giving money to via subscriptions down the line.

And yes there have been companies I have tested for who have very much shot themselves in the foot with thier attitudes and approaches that I have experienced during testing and others who have very much set themselves above and apart from the crowd.

I have always said testing IS a responsibility not a right.. it is work and yes it is npaid, but it is certainly not work without benefit or prupose for a tester.. it gives us what we need, if we are bothered to look for it and care enough to contribute to the process in order to get it.

That also means I have always had a fairly low opinion and very little respect or sympathy for those who join betas simply to play and contribute and take nothing from the process.. they b*tch and whine about the state of the game ignoring the fact it is in BETA for a reason, all the while contributing nothing.. and leaving themselves sold short as a result.

Posted: Aug 6th 2009 4:37PM mysecretid said

  • 2 hearts
  • Report
Are betas even about actual testing any more?

I mean, lately, they seem more like "free preview" releases meant to drive hype for the game's release.

I don't mind helping out to make a game I'm interested in work better -- what irritates me is participating in a beta "test" where you're not really asked to do any sort of testing -- just sell the game to your friends, and (randomly) report any minor bugs you might notice.

Posted: Aug 6th 2009 11:08PM CalebG said

  • 2 hearts
  • Report
Throughout the betas that I have signed up for online and been given the 'honor' to participate in (In reality, I probably was lucky, or selected based on criteria like location or computer specifications), I have never been paid a single cent.

Do I see myself as being exploited as free labor? Not really, I understood that when I signed up, I believe it is stated in one of the many TOA/TOS/NDA forms that most people just don't read now a days (I don't blame them really).

It is simple at the end of the day. If you don't wish to be 'exploited', don't join the beta! Or work in a company that would put you on their payroll for being one.

Know that joining a beta has certain benefits, which value varies from person to person;
You get to try out the game before others.
You have a hand, no matter how slight, in shaping the game into what would be at release.
You are assisting the developers in hitting their expected launch date.

Now if you were interested in the game, and are planning to play the game at launch, I assume all 3 points are equally as important.

I have a day job, and I beta test for the above reasons. If companies would start giving out monetary rewards or gifts (and some already are) for beta testing, that is great, but if not, I for one, wouldn't stop applying for betas.


Featured Stories

Engadget

Engadget

Joystiq

Joystiq

WoW Insider

WoW

TUAW

TUAW