Linden Lab having a bad day

Someone's playing with the circuit box.

Honestly...you'll post every little friggen quip about SL, meanwhile FFXI is only a few days away from the next xpack going live, fancon going on now, 1up.com posting a huge newstory about it, playonline.com updating with some pretty substancial news about the new conquest...and nothing. I love the site, and just hope you start to give the older games a little coverage. Give us anything! Please!

Or you might as well change the name from Massively to "Second Life: The Official Blog"

Thanks! I'm not a hater, just want to see this site live up to what we all know it can be.

We each have our specialty areas. Others are hard at work on news for other MMOs. Everyone's just getting up to speed.

Actually, we've got the whole team here from Second Life Insider so that's a few of us who have been full steam on SL news for more than a year - and yes, a lot of SL users come to us for SL news before or instead of the Official SL Blog.

Meh @ Xan.

If anything, I think the other news is a waste of blog space. The place was a LOT better when it was still simply the Second Life Insider.

What do I friggin care about the next 13 in a dozen romping stomping MMORPG?

Well, all the MMOs and VWs have various things in common - scaling issues, downtime, bugs, patches, load, lag, developer-to-user communications, griefers, economics, copyrights - pick a few.

The various communities have a lot more in common than each one just looking down on all the others ;)

Once again I have to say the problem is stupidity, LLs stupidity.

Linden Labs uses VPN between Collocation facilities. VPN has several issues:

1. Over head: packet size is bigger, plus encryption, encrypting and decrypting

2. You are dependent on the internet, while amazingly robust for some applications, amazingly week for others

3. No QoS across the network.

Most responsible businesses use MPLS, Direct Fiber, ATM, or even Frame Relay. Frame Relay and ATM are a bit awkward, but MPLS and Direct Fiber give you a good amount of bandwidth and all of them give you a reliable predictable performance! Done properly LL would have MPLS (up to a gig per pipe can be done) between locations with a gigabit VPN backup.

This also is a source of lag. Everyone looks to the sim for the lag, truthful, it’s not the sim, and it’s the network. We know this because when lag is bad across the grid a single naked person on a sim has lag. Lag is something that can build up on you with poor network design. Each packet that is not received generates an incoming packet, and then another out going packet, increasing traffic. So the more lag you have, the more lost packets, the more lag, and so on.

But what does this have to do with VPN, very simple, no QoS, you have packet issues between collocation faculties, you generate more traffic and it gets worse.

Until Linden Labs addresses its core network problems, changes to the clients, sims, and database farm will only have minimal affect on performance.

Lastly let me say, Linden Labs is full of developers and programmers. They are very good at what they do, develop and program. What they suck at, and I’ve seen this at several companies is run an technology infrastructure. They need network administrators in there, running the network and doing things right, like the big boys do.
Arthur Fermi

Not forget to mention that people aren't too bright about using SL in the best manner.

There are people that uploads bunch of 1024x1024 textures for little things....

There are people that wrote a very very bad script that causes insane amount of lag in the sim.

There are people likes to wear bunch of huds and blings for no good reason....

There are people using group to spam non-stop....

There are people using SL's database calls repeatedly over and over.

There are people using sim's resources to griefs back with someone...

Scaling Issue? Quite the problem... but really, people need to
learn how to use SL better to help LL's with the scaling issue.

http://blog.secondlife.com/2007/08/11/more-info-on-this-weeks-grid-problems/#more-1140

You tunnel across the internet, they are using IPSec VPN. You do not tunnel across private circuits, you might encrypt, but there is no need to tunnel as the pipe is yours, and only yours, and using a private dedicated circuit between locations you also have QoS. There is no QoS over the internet. Technically speaking the only thing they would really want to encrypt over private dedicated circuits would be credit card and RL information, but that should probably be on a completely different communications link.

I can give a perfect example of the problems with using the internet for your traffic between locations. We had 10 locations that were connected via VPN over the internet. When the Starr Report came out (Monica Lewinski for those who are younger) there was so much traffic on the internet it brought out locations down!

Until the LL decides that QoS is important, we will continue to have problems due to the network design.

But they are using the internet as thier link, they aren't using private dedicated circuts, traffic over the internet is the least stable form of communicaton.

If you look at Cisco internal private network you will find that it is primarly dedicated provisioned circuts, it was a mix of frame relay and ATM between offices. This is the best way to ensure QoS across the network.

They were talking about VPN over the internet at some point, I will try to dig it up. However, if you were just talking encrypting data over a private network, why IPSec VPN tunnels? Why not just encrypt the data on the way out, the tunnel adds nothing to security, just over head.

I would love to see that network design, because I would think you would have a problem with broadcast traffic across the network since you would have matching subnets, vpn or other wise. If you have different subnets, then it would be easier to connect between them via a router, or if it were a performance issue a layer 2 swtich ...

But back to Linden Labs, they obviously regardless of topology have some serious networking issues, that are causing the sim performance issues.

Definitely there _is_ an issue. From what we've experienced to-date the colocation facilities seem to be almost entirely dependent on each-other. It never seems to be a case of "lose contact with a colo and just lose those sims" - asset chains back up, logins bog down, underlying services backlog and fail.

All that suggests that adding more colocation sites would increase the frequency and severity of failure.