My Masters Dissertation on Internet-Scale Virtual Environment Architectures October 25, 2010
Posted by justincc in hypergrid, opensim, opensim-arch, secondlife, virtual-environments, virtual-worlds.trackback
Hi folks. For the past couple of years, as well as working for IBM and then on various OpenSim-related jobs, I’ve been doing a part-time Masters in Software Engineering at the University of Oxford. As part of this degree, as well as completing assignments for various taught courses, the student has to conduct an independent project and write a dissertation about it.
Naturally, I chose virtual environments/worlds as my project area, with OpenSim as the chief exemplar :-). More specifically, I decided to do some thinking about the possible architectures for creating a truly decentralized Internet-Scale virtual environment, as opposed to the classic ‘grid’ OpenSim/Second Life model where simulators are distributed but services (assets, inventory, users) are centralized. Naturally, initiatives such as Crista Lopes’ Hypergrid play a large part in these considerations.
I’m now making the dissertation publicly available.
The initial part is devoted to describing the classic ‘grid’ model, both conceptually (e.g. through the lens of the dimensions of transparency for distributed systems defined by the ISO International Standard on Open Distributed Processing) and in formal Z notation. The second part of the dissertation takes this description and compares it against what I think are the requirements for a truly Internet-scale virtual environment network. The last part of the text explores alternative architectures to the classic grid model for getting to Internet-scale.
Some of the models are significantly simplified from their real-life implementations. For instance, the OpenSim model presented consists of only four services (asset, inventory, user and grid). The Hypergrid model is based on Hypergrid 1.0 rather than the more recent revisions. In the main, these simplifications were made for clarity of argument whilst hopefully leaving the critical architectural features intact.
The dissertation also doesn’t take into account approaches that are radically different from OpenSim’s client-server system, such as OpenCroquet’s peer to peer synchronization architecture. I’d loved to have written and thought more about this but I simply ran out of space and time.
Nonetheless, I hope this dissertation might prove useful to people thinking about the future of Internet-Scale virtual environments, if only as a springboard for their own explorations. Naturally, I’d be very interested in any comments or questions that people might have.
p.s. For anybody wondering, this final bit of work was enough to secure me a pass with distinction ;-).
justincc's opensim blog 
Congratulations om on the MSc Justin… and thanks for making the text available.
CONGRATULATIONS!
Justin —
Very cool! Great job! Does that mean that you’ll now have more time for consulting work? Or launch your own company? 😀
We can only hope!
Congrats!
— Maria
Big congrats Justin! And thanks for all the much-appreciated work in OpenSim too.
Really looking forward to reading the dissertation Justin and hoping I can understand at least part of it 🙂
I keep saying we’re incredibly lucky to have thought leaders like yourself so deeply involved in Opensim. You guys don’t get half enough thanks.
Congratulations on passing with distinction. That’s a serious win in academic circles.
I can’t wait to see what you do next. 🙂
Congratulations!
And thanks for the freebie 😀
Those graduation projects are a pain, but worth it. You learn and there’s almost no classes on the last period :p
Nice!!!!
Congratulations Justin!
Thanks very much for the congratulations, everybody. It’s really nice to have it completed at last – part-time degrees are always a challenge 🙂
@James – Hopefully most of it is understandable – I always try to write in an accessible manner. The only parts you might want to skip are the descriptions of the model in Z notation – these are probably pretty incomprehensible unless you’ve studied that formal notation.
@Maria – If only I had more time – all my consulting hours are still currently being taken up with Intel / Sandia National Labs work. And other offers come through pretty regularly.
But in truth, I’d love to work a lot more on core OpenSim stability and features. What I really need to do is to start pitching the business plan/vision to help build out our embryonic Internet-scale metaverse 🙂
Justin,
Congratz, I liked to read it – good style, well defined scope, and clear concepts.
Would be nice if, now you did the scalability part – very good to compare VE with WEB, that made the perspective realistic -, could write about IP (intellectual property) and security. I guess those are the issues why services handled by the (propriety) SIM are in favor (at this time). I do not see how to handle these on the client side with the same trustworthy Indeed trust is the thing.
Anyway good paper.
Justin,
I forgot, are we waiting for (another) fat client revival?
Congratulation
[…] worlds could scale in a fully distributed and completely interconnected way, take a look at this: My Masters Dissertation on Internet-Scale Virtual Environment Architectures | justincc.org Written by one of the core devs of OpenSim. __________________ Be Cunning and Full of […]
@Ulysses – Thanks. Yes, there’s a lot that can be written about IP and security. I found myself doing some of this until I realized that it would take absolutely ages to write on these topics to any reasonable standard. I think is a very complex area that will take a very long time to shake out.
As regards clients, I think that it would be really interesting to see one that allows you to change the user interface much more dynamically. much more like a webpage than fixed window menus. In some work I’m doing at the moment for a serious game, I’m having to use the built-in ‘media browser’ to display arbitrary menus and get input, which is a clumsy solution at best since everything appears in it’s own heavily bordered window. I know media-on-a-prim exists and might end up being a better solution, but unfortunately there’s not enough scope to migrate to Viewer 2 right now.
I can imagine that a 3d virtual environment client could eventually be most commonly found as part of a web browser. However, I think the integration with a webpage needs to be much more fundamental than just having a client running as a flash plugin. Maybe we’ll see this in HTML 10 🙂
Very nice Justin. And it was nice of you to post this for us to look at.
Keep up the excellent work!
–rjs
w00t! Congrats Justin! You can breathe a sigh of relief now. 😉
[…] OpenSim grids will, and do, have firewalled intranet-like portions and also external hypergrid-enabled parts. For example, universities have their own websites hosted by their own IT departments, why would they choose a community grid to host their virtual worlds? A community will emerge from hypergrid technology as Justin Clark-Casey presented in his Oxford Masters dissertation. […]
[…] OpenSim grids will, and do, have firewalled intranet-like portions and also external hypergrid-enabled parts. After all, universities have their websites hosted by their own IT departments in their own data centres, why would they choose a third-party community grid to host their virtual worlds? True communities will emerge from hypergrid technology just as Justin Clark-Casey presented in his Oxford Masters dissertation. […]
[…] JustinCC: My Masters Dissertation on Internet-Scale Virtual Environment Architectures – “The initial part is devoted to describing the classic ‘grid’ model, both conceptually (e.g. through the lens of the dimensions of transparency for distributed systems defined by the ISO International Standard on Open Distributed Processing) and in formal Z notation. The second part of the dissertation takes this description and compares it against what I think are the requirements for a truly Internet-scale virtual environment network. The last part of the text explores alternative architectures to the classic grid model for getting to Internet-scale.“ […]
Nice job! Just started reading the dissertation, and already enjoying your insights. I’m just barely starting to seriously implement OpenSim, so the technical background information is especially useful.
Great job! I’ve just read it this morning in one go, and loved it! My own research pathways have started to look into such issues, and I’d love to chat with you. I’ll be in Oxford most of the week, so perhaps that could happen (just google me and you’ll find my e-mail, if you’re available).
Thank you for sharing your dissertation, Justine and congratulations on your pass with distinction.
I found it to be a clear and interesting insight into the architecture of Opensim – Z notation aside, which I am not familiar with – and analysis of the problems associated with developing an open meterverse. I have to say though it left me somewhat despondent that such a metaverse can ever come about with any real security built into it. From what I know of Opensim those who build virtual environments on the platform have roughly divided up into three camps. The Education camp clearly sees financial advantages and the control over the grid Opensim offers which enables them to protect their students somewhat better than in SecondLife. Another group prefer to wall themselves up into proprietary grids in order to protect content creators and, no doubt, hold on to their traffic which makes them no different from SecondLife in terms of business model (other than being cheaper of course). And a third camp which embraces Hypergrid and seems willing to live with the security issues and lack of content protection in order to enjoy inter-grid teleporting. One might call the third group “sharers” willing to share traffic, content (some of it copy-botted) and somewhat less desirable parasites like griefers.
I would agree that the Direct client services architecture dose appear to be the most promising approach to developing a truly open metaverse of hyper-connected VEs but it strikes me at the same time as a lot of work changing the fundamental architecture of Opensim. Even if the will is there I imagine it will be years in the making which could see it out-dated or overtaken by other softwares before it can come about. Even Linden Labs has said in the past they may open source their serve code eventually. However, I know that Diva Canto shares the open worlds dream and she may yet come up with a workable solution in HG 2. Still, all that said, it is only the developers of Opensim that I know of that really talk about an open metaverse (apart from OpenCobalt perhaps) and your study did, at least, reinforce that.
Thanks for your kind comments, Gaga.
I think that you’re right – an open metaverse will be many, many years in the making – we’ve only just begun the journey and it’s a far from simple quest. But I think there’s going to be a helluvalot of interesting stuff on the way 🙂
I’m not sure there’s a strict division between your 3 groups. I know many educators, for instance, who are very interesting in Hypergrid. And there are other groups as well, such as people who want to use OpenSim or virtual environments in standalone applications, and those who want to build a dedicated community. One of the great things about OpenSim is that it opens up great opportunities for experimentation, in all kinds of ways, to see what this stuff is really good for.
And to be honest, I think that the more competing software trying to get to this destination, the better. The more people working in this space, the more popular it will get and the more ideas will be generated. If something comes along that does the job better than OpenSim then I’ll be cheering it all the way (though I think the fundamental infrastructure has to be open-source in this day and age).
Thanks Justin and Leonel, glad to know that you found my scribblings useful 🙂
i finally read your dissertation and have been carrying it on my keychain’s USB . . . when i read about the copy on write it made me think about objects and instances of them
from your paper (very nice to read meaning you write in a pleasant and approachable manner), would it be possible to do something like i outlined here: http://iliveisl.com/update-one-to-update-many/
it seems like it would be somewhat similar to an avatar’s attribute’s. but perhaps i am missing a much bigger resource issue in my thoughts
thanks and very well done on your paper, my understanding of it all improved greatly =)
Congrats on your hard work i do hope what ever path you take in this life it will be one to help others and not be trap by greed like so many have.
For greed is what is killing the USA and all our people will be hurt do to it.
you have great skills and it can help so many in this world may be people can not see that as of yet but even the people working in opensim to help others are helping so many with out knowing it.
learning never ends and so many in this world are left with out some one to help them learn.
we need better teachers not just for the young but for the older people that no longer have jobs do to lack of new training.
even our great veterans coming back from the war needs help and many of them needs jobs.
if you become a teacher you not only help your self but many people.