AndrewWillmottSims2Talk.md

Andrew Willmott Sims 2 Talk

This is an auto-generated transcript for Andrew Willmott's talk Shipping Sims 2 from GameTech 2004.

• talk link: https://www.andrewwillmott.com/talks/shipping-sims-2

• mp3 link: http://game-tech.com/Talks/Willmott.mp3

transcript.txt

All right, we've got one more game-oriented lecture coming up, and it's given by Andrew
Wilmot of Maxis about The Sims 2, which should provide a pretty interesting contrast to the
previous games, which were all shooters about shooting things and stuff.
So go for it.
Yeah, this was quite a different game.
And it's also by far the most content-heavy game I've ever worked on.
So what I'm going to do is mainly give kind of a breadth-first overview of the entire
process, and just try and hit as many different parts of the puzzle as possible, rather than
drilling down deep on anything too much in particular.
So in questions at the end, I'll be happy to go into detail on some stuff, but the main
theme of this talk is just to try and give an overview of how you put together what was
really a pretty massive game.
So yeah, why is Sims 2 interesting?
A lot of animation.
Like some of the other talks here, a lot of sound, surprisingly.
Massive amounts of people.
We've had at least probably 250, maybe 300 people touch this project.
That's not all at once, but still.
It's interesting from the point of view that the users are our level designers.
You know, this is completely different from the normal FPS process, where you have pretty
much the whole point of your content is to produce levels.
So our users are designing the levels, we're just giving them the content.
And our simulator is kind of interesting in that the whole thing is driven by a visual
scripting language.
So the Sims 2 was a long project.
We started in late 2000.
For a long time, it was just a pretty small research team basically doing constant demos
and constantly losing people or gaining people from the rest of the studio due to other projects
going on.
We were in full production for about two or three years.
A lot of people at the end, as I said.
And we had a pretty big slip.
We were trying very hard to hit a particular deadline, and we missed it.
So there's also some interesting stuff there.
The downside of that for us was that we had an extended crunch period.
I'm going to give more stats as we go along through each particular section, but these
are like the main ones.
For me, the 11,000 shipped animations is the really big one.
A lot of the other kinds of data go into a game, but the 1.1 million lines of code quoted
there is no blank lines and no comments.
I'm just going to quickly flip through a couple of diagrams, which you're not expected to
memorize, just roughly showing what the process was on Sims 2, so that as I go through the
sections a bit later, you have something to hang your thoughts on.
This is basically how our art and object scripter process kind of went through the pipeline
and produced mostly either scripts or 3D or 2D resources.
On the other side, we had code picking up these resources, and our architecture was
basically that we had kind of a platform, a game-neutral rendering engine, and then
a diverse bunch of components basically making up the application layer, mostly driven just
by object scripts, text scripts, and binary resources.
A lot is like a saved house, a saved file.
I'm going to run through a few of the more interesting areas of the Sims 2 process, and
at the end I'm going to look at the lessons we learned and how we're moving forward with
some of these lessons.
So we're starting with art.
In many ways, a lot of what we try and deliver is basically the output of our animators as
entertainment to our users.
I mean, obviously that's not everything, but it's a significant fraction of it.
So you can kind of view a lot of our pipeline as a kind of content delivery system.
So these are some stats from art.
A lot of models as well as animations.
A lot of effects, which we'll get to a bit later.
Basically just a lot of data.
This is somewhat less interesting UI.
I'm actually not going to talk about UI too much because it ran pretty smoothly.
It was, we basically have UI down as a problem at this stage, which contrasts quite heavily
to some of the other parts of the game.
So we had a lot of artists.
Our technical art director likes to tell the story that, he comes from Pixar and he worked
on the first Toy Story movie, and the first Toy Story movie actually had fewer animators
than we did on this project.
So that's kind of giving you an idea of where games are getting compared to earlier computer
animation.
Our art tools were pretty vanilla.
So Maxis had been kind of a Maxis-based studio before this title, and we, Sims 2 was really
the start of the whole Maya transition, and now we're mostly Maya-based.
And a lot of that is just because we find it a better tool for animation.
Basically we bought over all of the animations from Sims 1, which were in Max format, wrote
a complicated pipeline to kind of drive re-rigged animation in Maya by the source animations
imported from Max.
We didn't use any of this stuff in the end, but it was pretty useful as kind of like an
initial guide.
The reason we didn't use much of it is just that some of the animations were no longer
relevant and everything that was basically got redone to a much higher kind of level
of detail.
Like some of the other previous talkers, we used Null a lot for basically all of our UI
within Maya, and it did a very good job of that.
Our pipeline, again, is reasonably straightforward.
One interesting thing we did is about halfway through the project, added direct read support
for Photoshop, and what this allowed us to do is have the artists work on all of the
layers for, say, a sim or a model in a single Photoshop file, which helped their process
a lot.
In terms of animation and modeling, basically we went from Maya, we used a EA internal kind
of platform neutral export format, and then we had our own asset compiler called Go Disco
that converted that into game resources.
Now the way this all worked was kind of interesting.
The artists basically checked directly into Perforce, and the reason they didn't forget
to check into Perforce is that was basically, there were two parts to that step.
You checked in your new animation or model or whatever, and then that would trigger the
build machine that would then convert that content to the game format, and you'd get
an email back telling you if there were any errors or whether the process had gone smoothly.
So you weren't done until you got that email back and it was all clear.
This whole process also built a big web page with basically a link for every single asset
in the game.
So you could browse this website and drill down to any particular model and all of the
game events that might be in it, or number of vertices, you know, you name it.
The Sims 2 skeleton increased in complexity quite a bit over Sims 1X.
Sims 1X actually had smooth skinning, which I was initially surprised to find out about.
A lot of the extra attention to detail went into the facial morph targets, is where we
got a lot of our extra kind of emotion and expressiveness, which is a bit dark so you
can actually see it by the game, I guess.
So the main skeleton had 64 weighted bones that were actually being processed by either
the GPU or on CPU, but the skeleton itself had 116.
So a lot of these extra bones were for grips, you know, where to put your spectacles, where
to put your hat grip for matching up IK, and all sorts of things like that.
And it was kind of because of all those extra slots and so on that we didn't lock the skeleton
in the end.
So this was a big focus of ours early on with Sims 2, it was like, you know, finalize the
skeleton, get the number of bones and the layout and everything absolutely rigid, and
then we'll lock it and forget about it.
But that basically didn't work because as design evolved, we needed to add, you know,
grips, remove grips, modify our skeleton.
So instead we had a system whereby all of our animations referenced one master Maya
file that contained the skeleton.
And we had an indexing system that made it reasonably easy to kind of add bones or even
remove them without invalidating all the animations.
So modeling, we're actually lucky enough to have a lot of the Sims 1 modeling modelers
come through, which is kind of important to the Sims because, you know, they developed
a particular look in Sims 1 that we were basically trying to amp up, and they also had a lot
of experience and ways to handle texturing and modeling on an animated character when
you have, you know, hundreds of skins maybe.
There's a certain amount of process there that has to run more smoothly than with a
fewer number.
One of the things that went wrong was that the graphics team didn't really give artists
either a poly budget or a texture budget.
This was very kind of hand-wavy, you know, that the code will take care of it, will either
write dynamic simplification.
What we actually shipped, did manage to ship with respect to the textures was that basically
we dropped mid-levels from all of our textures that are currently being referenced until
it all fits within video RAM.
But that's a pretty kind of clunky and error-prone process, so I don't think it really worked
out.
We contracted out our object LODs, which seems to be the thing to do these days.
Didn't work as well as you might think because there's a lot of overhead in contracting out
our, in terms of getting the external team up to speed on your process.
And if you don't get that exactly right, you wind up having to fix every animation or model
that comes back, and we had a fair bit of that.
So because we had a lot of models, we implemented a bunch of different visualization modes to
try and spot kind of bad model usage.
So we had textual density visualization, so we could try and figure out where we were
wasting texture memory.
We had a neat little animated model display where basically the content viewer that we
used to view our models or animations would repeatedly lurk between, say, a model and
then its layout in the texture map, so you could kind of see at a glance where the texture
map was going on that particular model.
Obviously late in the game, we also had to implement this thing to let the artist switch
between the different shader parts in the game.
This was kind of because the shader parts were so buggy.
We wanted to make sure that they were tuning the lighting and so on to the right thing.
Speaking of lighting, this is one of the most well-processed parts of the game.
The art team actually produced a quite extensive TDD on the whole lighting design, and we even
prototyped it beforehand.
We had two lighters during the course of the game, kind of one before slip and one after
slip, and we pretty much had dedicated people for that because rather than tuning lighting
for a level or something, they were tuning a system that had to try and light the game
no matter what the user did, no matter where they placed their lamps or built a house or
anything like that.
So that was kind of a new challenge for us, and some of it worked, some of it didn't.
Just quickly on what we did for level of detail, so this is a lesson in kind of talking about
something a lot and not actually sitting down and implementing it.
We wound up with static LOD, never formally spec'd.
All of our sims had at least one LOD with fewer bones and about half the poly count.
And in fact, we don't even switch LOD dynamically within a lot.
I didn't quite know where to throw this in.
This is sound.
Maybe it's art, maybe it's engineering.
We had a lot of sound, like some of the previous games.
The stat that always gets me is 43,000 Vox samples.
And the reason that's kind of mind-blowing is that all of our sims speak in simlish,
so we don't even have to localize that.
So if you imagine having to localize all those Vox samples, it would just be insane.
Lots of footsteps and just a lot of ambient sound kind of triggered off the camera and
triggered by various objects and so on.
And sound was always, at one point sound was actually two-thirds of our data footprint,
using various compression methods.
We got that down to a gigabyte.
But it's a big part of our game.
Okay, design.
This was like an early version of The Neighborhood that thankfully was replaced by something
much prettier.
Design was one of the biggest areas in Sims 2.
It was really difficult, basically.
You're trying to improve on Sims 1, which is one of the best-selling games of all time,
and attract people who haven't played the game before, bring back people who played
it for a while and got bored, and not lose all the people who are still obsessed about
the original or buying all the expansion packs.
Also, we have surveys showing that people play The Sims in very different ways.
It's almost an even split between four different styles of play.
I can never remember all of them, but it's something like there are people who play the
romance game, you know, the actual simulated bit, people who just build houses, people
who basically like to torture their Sims, that's a big one, and something else.
So we set ourselves a goal about two years out from ship four to get a 90-plus Metacritic
rating, because this was a big thing at EA at the time, and I think still is.
If you can get that rating, then your sales go up by a certain factor of X.
And we actually hit that, you know, we're still at 91, I think.
So there's all of that, and then there's the pressure that comes from a high-profile title,
which means a lot of constant demo pressure, so I don't think the Oddworld guys suffer
too much worse than us in terms of, like, constant demos and having to deal with...
So while we're not selling to external producers, we are selling to EA executives, and we have
to keep them on board with our changes and so on.
We have this constant question of, you know, we're throwing new stuff in there, trying
to make it compelling, where do we stop?
If we take too long, then we basically have to throw more stuff in, because it's already
been too long since the last one.
And early on, we also lost a lot of people and a lot of ideas to, well, the expansion
of Simpson's 1X.
So new gameplay.
We started off with movies as the main idea, you know, let people do the machina video-making
thing.
Yeah.
And we actually came back to that as almost the last idea.
We brought on aging, the whole idea of having a sim grow up through its life cycles, having
four different age brackets and doing transitions between them.
And then we threw another thing onto the pile of generations and genetics, you know, family
trees, breed, crossbreed your sims and see what they look like.
Horrific, really.
Then, big life moments that we were trying to bring in, you know, some of the oomph that
other games get kind of out of, you know, you defeat the boss monster and you get some
great cutscene.
So we were trying to emphasize some big moments in the gameplay.
Aspirations and then wants and fears was pretty much the last one and that's probably, I think,
the most compelling new bit of gameplay.
So we kind of drew the line at that.
So wants and fears basically and aspirations I think came after the slip.
So once we'd slipped and half of the reason we slipped was because we didn't feel the
design was there.
We needed extremely tight process control to make sure that we didn't slip again.
So we had very strict kind of change review.
Anything that was going to be changed about the game had to go through meetings.
We dedicated people to particular features and we basically had SWAT teams to try and
bring these new features, design additions to completion without screwing up the currently
working part of the game.
So in the end it shipped.
That first point is kind of redundant.
We just could not have shipped at the original time.
We probably couldn't even have put anything into a box.
We got enough crucial extra gameplay, the wants and fears thing I think was crucial,
and we also got the chance to finish engineering which was the other reason that we slipped.
We were way behind on engineering.
And in the end it was quite a success.
So the lesson from that if you kind of go back at Maxis a year is never give up because
things were looking pretty bleak at one stage.
There's always a cost when you slip.
So basically having to constantly redesign and we'll revisit this is always a negative.
Design changes are really costly and all of the post mortems I read about this project
were basically talking about how difficult it was to deal with constant change.
We also got sucked into the EA Redwood Shores mothership early 2004 after the slip.
I think they wanted to keep a closer eye on us.
And there were various kind of negatives that came out of that.
So we have a very big incentive to kind of learn how to do this kind of thing better.
So object engineering.
So this is probably as important as art.
These are the guys who basically do all of the gameplay in The Sims.
So the way The Sims works is that there's a simulator running a bunch of objects, you
know, objects on various tiles and each object has a whole series of associated scripts and
is also responsible for sequencing all of the animations that involve that object.
So mostly this continued to work out well.
I mean, the simulator is a reasonably nice process.
We basically have it done in terms of adding objects and so on, you know.
We've built up a lot of experience from the expansion packs and so on.
There's still some problems with it.
The scripting language is pretty simplistic and often what we're trying to expose to it
from the game is reasonably complex.
So you wind up with some pretty hairy scripts, which is what I'm referring to in that mismatched
line.
Another thing that goes on is this is the way we do our blending, as I mentioned before.
So the artists basically provide all the source animations and then it's the object engineer's
role to kind of blend those together to produce the final animated result.
So you've got an obvious problem there of synchronizing, especially if the OEs aren't
sitting right next to the animators, which was our situation.
So at least early on, you constantly had object engineers going back to talk to the animators,
trying to get them to run through, you know, show them in Maya where certain events were,
trying to figure out what they were missing.
It was a big process problem.
As a result of this, one of the OEs actually, this Skunkworks project, produced this little
tool called Clockwork, which basically sucked in all the animations, read all the tags and
provided a little browser, which made it very easy to figure out what the animations were
doing and where all the timing events were.
So Edith is the visual scripting language that drives all of this.
And actually, one of the main learning experiences from The Sims 2 and probably the previous
expansion packs too, is that for this situation anyway, visual scripting just does not work
very well.
You give up an awful lot for a nice visual scripting language.
There's no revision history in Perforce, no good search and replace, all kinds of things
you just take for granted with script text scripting languages you lose.
And that becomes pretty frustrating.
So I did a little survey at the end of the project and pretty much, well, to a man, all
OEs wanted to move away from it.
The one good thing about a visual scripting language is that it pretty much enforces having
a good debugger, although I gather it was broken for a good part of the time.
So we even tried to use Edith a bit on Sim City 4 of all things, which didn't work out
at all well.
So our studio can say this now, except we're basically going to ditch Edith and attempt
to use Lua in the same capacity.
Engineering.
We had about 28 engineers at peak, starting off with about 5 or 10 for the first few years.
So as I said before, 1.1 million lines of code.
We have quite a lot of shared code at Maxis.
There's this framework code that has been shared, I believe started with SimCopter,
and has been used in all sorts of titles since then, SimCity 3000, SimCity 4, Sims Online,
the original Sims, all the UI code came from this framework.
And we also had a lot of engineering-generated scripts, mostly materials, but also things
like cameras and catalog and light tuning.
We had one dedicated engineer to basically what we call the WorldDB, which is the world
representation, so the train and so on.
And that was our main link between gameplay and the engine.
There was a lot of talk early on in Sims 2 about moving away from the tile-based system
that it subsists on, and quite frankly, we go through this with every new title, and
it's the same with SimCity.
All of our games are kind of tile-based, aesthetically you'd like to move away from that.
The reason that we stay with tile-based systems is just UI and kind of gameplay.
We would have to drastically rewrite the simulator to be able to move away from that.
This actually gave us a problem later on, because we had this nice generalized WorldDB
code, and it was quite hard to make it efficient in terms of a tile-based structure.
So routing was one of the biggest complaints about Sims 1.
You had this party syndrome where you'd have 20 people in a lot, all trying to get somewhere
at some time, and basically stacking up.
Early on, we contracted a company to write a replacement for the Sims 1 router, but the
results weren't that fantastic.
And even if they had been, there was still a problem in that the router is really essential
to gameplay in The Sims.
So it's really something that we needed to own and kind of iterate on with the simulation
engineers.
So instead, we dedicated an engineer to that, and I think that was a very good decision.
That's just a quick list of some of the features in it.
Probably what makes it most different from other routers is just some of these, the last
two extra features, and also the fact that the routing paths tended to vary a lot based
on what the simulator was telling the routing system.
So there was very kind of tight coupling there.
Something pretty standard, blended animations, two bone IK, and a bunch of extra features
to handle certain things.
Standard reach is probably the biggest one.
We obviously have the problem of Sims walking around and having to pick up cups and put
stuff in the microwave and all the rest of it.
We also had an effect system, which started off as just a particle system on SimCity 4,
and it's now kind of grown into this monster prototyping system whereby, so it's all script-based,
hot-loaded, and kind of hierarchically composed.
So a lot of the stuff in Sims, the dynamic geometry that you see is almost all the effect
system, and that kind of includes things like the thought balloons, a lot of the UI that
isn't text-based.
So I just have a little example here of, say, the fish.
So if you play the Sims, you can buy this little fish tank, and the little fish in the
aquarium are all run by this effect system, basically a combination of random walks and
colliders and quite elaborate state transitions.
We tried to move forward the neighborhood quite a bit on Sims 2.
In Sims 1, the neighborhood was screen capture, essentially, screen render.
We tried to make it much more of like a sandbox in Sims 2, much like the lot.
So the hardest technical problem there was just kind of imposterizing lots dynamically
on the fly.
When you exit from a lot, then we build a little imposter that kind of captures roughly
what it looks like, and we also import all of our terrains from SimCity, which made production
actually of, you know, a variety of different neighborhoods pretty easy.
There's nothing like having one of your other games provide quite an elaborate production
tool to speed things up.
I talked before about the lighting system and how it had to handle, basically, a dynamic
environment.
How we did this is that lighting was room-based.
We had code that would break up our buildings into rooms and portals between each room,
and then we basically just did the obvious things.
The most elaborate part of the system is just that portals can transmit light.
So you can see in the left-hand picture here that actually the only light that's being
specified here is that the artist has, lighting tuner has specified the outdoor lighting conditions,
and then the light makes its way in through the portals all the way into that inner room
and a set of fixed function lights is constructed there to kind of light those chairs correctly.
So yeah.
Early on we decided we absolutely needed over-bright lighting to get, you know, a decent look.
That actually bit us later in a non-obvious way.
So basically we had a lot of problems when tuning the lighting system, avoiding blow-out,
because of the way that windows could be placed almost anywhere and object lights could be
placed almost anywhere, preventing a sim from basically blowing out completely to white
or a nearby bed was problematic.
Shadows are, the only really tricky thing about the shadows was just the sheer number
of them.
So basically every object has to have a shadow.
The terrain and house were actually not done using traditional kind of graphics, projective
texturing shadows.
There was a fast CPU side algorithm that gave us back a little map that we could look up
for any location in the world, you know, is that in shadow depending on the terrain or
the house, which suited our target hardware.
So our target hardware was very much kind of DX7 level, so we had to keep most of our
graphics tricks pretty simple.
The other thing we used was these GUOBs, which stand for Generic Under Object Blobs.
These are basically just cards like you saw in the ATI demo that we pre-render to look
originally like blobs, but after that quite nice, you know, we did Maya software rendering
to get a nice kind of ambient type shadow indoors.
This is most important for things like contact shadows.
If you have a painting against a wall, it really looks much better if you kind of include
the contact shadow that it's casting against that wall, it helps kind of seat it.
So the graphics engine was a curiosity in some ways.
It was written as a completely generic engine that, you know, tried to disregard anything
to do with gameplay in the Sims engine itself, and it was written as a completely generic
scene graph.
So the way you assemble your scene is basically by creating normal, you know, Maya style.
Your Sim might be a hundred different transform nodes that you've read off disk and you insert
that into the scene.
If you want to find a node on that Sim, you do a traversal over that sub-branch and look
for tags and so on.
So basically a lot of caching went on to make this all work properly.
And it just seems like a fundamentally bad idea.
This kind of thing really belongs in your art pipeline, I would claim.
Because just the overhead and the extra complexity in dealing with a scene graph, you know, actually
in gameplay code was enough that I think it brought down the productivity of the actual
game programmers quite a bit.
So we had graphics performance problems with this, as you might imagine, and plus just
our batch count.
So we're rendering a lot with a whole bunch of, you know, hundreds, two hundreds of different
objects in it, and each object might have different subsets, and plus you have the walls
and the roof and all the rest of it.
So this was a problem for us early on, and eventually the solution we turned to was a
very old and hoary one.
So SimCity uses Dirty Rex.
Basically dynamic objects are re-rendered per frame, and anything that we know is static
we just hold over from the previous frame.
So I thought SimCity would be the last game that ever shipped using that technology, but
that's the way it went.
We had an initial kind of prototype system just to get working on low-end hardware, Dirty
Rex system, and this eventually morphed into over a number of months a full SimCity 4 style
dual-layer system, and this caused us no end of problems just because it was retrofitted.
There were all sorts of subsystems that basically needed a lot of extra code added so that we
could identify when things had changed and where they had changed.
Some of this came about just because our target platforms were so broad and so low on the
minimum ends.
So we tried to support actual non-TNL commodity Intel hardware, you know, the 815, 845, and
basically the broad hump of the cards we were trying to support or machines we were trying
to support were machines with DX7 level cards.
There's a whole bunch of extra slides and commentary in the proceedings that I encourage
you to read if you're interested in this, but given that a lot of this is kind of not
of interest going forward, I figured I'd better skip it.
Apart from this slide, so game configuration, this is a real headache, targeting that many
target platforms.
We had a reasonably elaborate system on SimCity to handle all the different possible configurations
and driver bugs and all the rest of it, and we basically amped that up by about a factor
of two, I think.
So our experience has been, I'm sure it's the same elsewhere, is that relying on cap
bits or any kind of, relying on the card to tell you what it can do or not is just, does
not work.
So basically we build a lot of logic into scripts that tell us, you know, what cards
can support which and where to set various level of detail parameters.
So memory handling, this is a PC title, so as usual we're pretty fast and loose about
memory.
A lot of STL, very similar to Charles' description of what they were using at Oddworld, almost
all vectors, not always efficient, but we can usually pick up cases where it isn't efficient
with profilers, and our main obsession with a lot of this stuff is just leak prevention.
Because we have to scale the game over across, across a number of platforms, it's much more
difficult to kind of hit, you know, we can't just hit a particular memory target, we probably
have hit four or five, depending on the different platforms that we're targeting, you know,
how much memory the system has, whether it's running a card that's going to require backing
from system memory, or the rest of it.
We also use ref counting a lot, and interfaces, and similar kind of smart pointers, auto ref
count is basically the same kind of smart pointer thing that Charles was talking about.
And for anything that really needs efficient allocation, we have custom allocators to do,
you know, similar pools, and things like that.
We have some interesting observations over the last couple of games that we've shipped.
Basically it's reasonably easy to track down leaks these days.
We've done a number of code sweeps, and kind of added up the percentages of where we find
the problems, and if you want to find a memory leak, just search for a new or delete in a
So as a result, we have almost no, we avoid this.
We used to have a lot of manual ref counting, we're replacing all of that, because that's
another hot spot.
Not quite as bad, but the trouble with manual ref counting is that it tends to be very fragile,
so we'll come along later and add some code, and break an existing working system.
And finally, we avoid the kind of ref count loops by having very kind of solid init and
shutdown approaches.
So basically be sure to init and shutdown all classes, and it's absolutely a requirement
that a shutdown releases all member auto-ref pointers.
I thought this would be interesting.
In terms of the biggest leak that we had during finaling, SimCity 4 was actually the Lua garbage
collection, which I find kind of ironic.
There was a race condition in the Lua code that led to massive memory leaks in a particular
situation.
In Sims 2, we discovered in the last couple of weeks that someone had left on a particular
logging system, and that basically that log data was all being accumulated in a big buffer.
Another of the interesting observations from Sims 2 is that we, although we got well back
from this, you know, by the time we shipped, we ran into situations as we entered finaling
where we were actually running out of virtual memory space.
So PC games have long used virtual memory as kind of like a crutch to avoid having to
be too careful about the amount of memory that you use, but that kind of free lunch
is running out.
Basically that address space is getting smaller and smaller depending on DLs and things like
that.
So I'll just give you an example.
This is a little snapshot of the Sims 2.
This is actually after we solved our big leaks that were causing us to run out of virtual
memory, but you can still see that big blue band in the middle is all of the allocation
from just our, basically the engine and the scene graph and the first layer of game code
over top of that, and we have a lot of allocation churn from that, and as a result, a lot of
kind of space that we've touched and then released again.
So resource management.
I'm not going to talk about this too much because I think it's been covered a fair bit
in previous talks.
We have a reasonably kind of conservative key-based system that we use at Maxis.
The only real wrinkle that came here is that the initial Sims 2 team, excuse me, weren't
really familiar with it, and first of all kind of overused resources, and second of
all tried to do an alternate kind of name-based system and wound up kind of hashing strings
into 32-bit UIDs.
Now you can kind of get away with that, and we do get away with it on, say, console titles
at Maxis where you have a smallish number of resources, 5,000, 10,000 or something,
but with our resource count, you're just bound to get collisions, and indeed we did,
and we also had the usual kind of custom content headaches of people, you know, someone in
India creates a skin, someone in England creates a skin, and then they upload their skins and
someone in America downloads them both, and you have to be careful about resource collisions.
So we basically brute-forced both of these problems.
The extending the instance ID to 64 bits, basically that solution came about just because
we'd painted ourselves into a corner, you know, with any amount of kind of simple straightforward
planning at the beginning of the project, we could have avoided the situation, but we
wound up very late having to deal with these collisions, and in the end making that kind
of brute-force change was the easiest thing to do.
Configuration management.
We're starting to get quite a big configuration management team.
It seemed quite similar to what Chris was talking about earlier with Bungie.
We had like six to eight engineers on our configuration management team, kind of shared
between Sims 2 and a few other projects.
We have a system at Maxis where we test the game by kind of running commands through a
command console, which has proven very effective.
So basically we can write a whole bunch of unit tests that run the game through its paces
and then check for certain conditions, and, you know, once we build up a library of tests
like that, we can be running them constantly, week in and week out.
Before each check-in, even early on in development, you're required to run a simple sniff test
to basically make sure that the game was reasonably functional.
We used DevTrack.
We had our own custom tool for rolling out builds.
So basically it turns out to be a bad idea to roll out development builds where you have
hundreds of thousands of files incrementally.
It's a bad idea for the network.
It's a bad idea for File.io.
We probably would have been better off just releasing zip files of daily builds.
We had a similar facility discussed before here where if the game crashed, it would send
off a stack dump and all sorts of other annotations about the simulator and the animation system,
and then these would be gathered and kind of published on a website.
So you could kind of go to that website on a particular day and see, you know, the most
common crashes and follow the links and actually get the place in the code where it crashed.
We had quite an elaborate source control set up.
So we actually had two branches.
The engineers worked on this DevLan branch, and that actually went through a testing cycle
before it was then integrated across the main line.
The main line was also then tested before that was in turn released through this Robby
tool to production and art.
So it turns out this is just way too much process for a game except when you're on the
last few months, and one of the common bits of feedback after the game shipped was basically
there was way too much turnaround between, say, an engineer checking in a feature and
an artist or a scripter seeing the result.
It could be up to a week, so pretty bad.
The other problem was just that I forgot to mention on the other slide is that our publishing
tool had no rollback facility.
So if we did, if in spite of all this process, we did manage to publish a bad build that
was broken in some way, that would basically stop people for a day or so.
So lessons learned.
So everything was kind of nice and peaceful after shipping, but at some point you have
to, like, sit down and figure out what you can change.
So the thing that went right was basically art and content like that.
We had all sorts of the usual process problems, but when it came down to it, we could always
hit art block.
And one of the things here is just that we have a lot of crossover with the animation
industry these days.
Like I said, our technical art director was from Pixar and was used to setting up a lot
of the pipeline systems that we use.
And we also are hiring from the animation industry now.
And they've been dealing with these problems for quite a while.
The things that went wrong in art are basically going back to design again.
There were a lot of late changes to design.
We had, I think at one stage, two or three art directors, and also executives and even
producers would throw in commentary about how something should look.
So there was kind of a lot of trashing going on there.
There were some communication issues, the usual thing with a team that big.
The tighter code turnaround that I was talking about, and just generally there wasn't a lot
of communication between the engineering team and art, apart from a few pathways.
The kind of person we're having trouble hiring is kind of technical art types.
So we're starting to see this need for the kind of technical director role from animation
again, someone who's kind of an artist, but is also very good at scripting or whipping
up little useful pipeline tools.
This was probably our biggest lesson.
It's just, the later you make a design change, especially if it's a reasonably big one, the
bigger the cost to engineering, art, production, just everyone.
So we were kind of between a rock and a hard place because we had to get that design right.
And it wasn't right.
You know, Maxis has been through this before with the Simsville title that never shipped.
The reason it never shipped is basically it looked gorgeous, but there was no gameplay
there.
So one of our biggest issues is basically how do you address that kind of thing, which
I'll get to a bit later.
Engineering lessons, we had a few.
The engineering process was pretty chaotic.
So we had a bit of a problem with, you know, a large section of engineering mostly concentrating
on the graphics engine and not on the game.
In fact, the game got neglected to some extent.
We had an ongoing problem, especially early on in development, of not having enough people
actually working on the simulator and the various things that were going to be delivering
all this new gameplay.
It's kind of indicative that the research team that went for two or three years, only
about half the time they ever had an engineer actually working on the Sims simulator code
base.
No one else was working on the graphics engine.
Like I mentioned before, a scene graph really belongs in the pipeline.
I can go into that in more detail afterwards, but this basically did not work for us particularly
well, just because it kind of sucks, well, there's the complexity issue, and then you
get into problems with fragility, where you're persisting, say, a scene graph branch that's
supposed to be matching, say, your code, your gameplay state, and if those get out
of sync, then you can run into all sorts of problems.
The resource problem and a few other simple things like that kind of taught us don't ignore,
you know, your existing shipping hardened code.
Don't think it's a good idea to spend engineering time trying to come up with a better system
where you already have a decent one.
Don't contract out core gameplay components.
That was the lesson from the routing.
And from engineering, basically more urgency, part of the result of having some engineers
kind of detached from the actual game is that there wasn't a lot of urgency, and that was
actually where most of our senior graphics, sorry, our senior engineers kind of fell,
so that meant there was kind of a lack of urgency from the top of the team to try and
get this thing out the door.
This is a transporter accident.
We have hundreds of these, usual kind of skinning accident.
So I'm going to run through a couple of slides of drilling down onto what things went wrong
in engineering, just because it's my area.
We had a lot of over-engineering, and I list a completely insane example here, but there
are other examples almost this insane.
So the concept of a pixel format, you know, ARGB, 5651, sorry, 5551, all the rest of it.
Something you'd normally represent by an enum.
Well, we had five interfaces, I think, and 2,200 lines of code, just to represent that
concept.
We had some people who are very enamored of C++ and of COM, and this kind of led to what
I call bubble wrap syndrome, which is where you have a reasonably simple kind of core
game concept or engine concept that you want to be able to manipulate, but has so much
layering over the top of it that it feels like you just can't push through and directly
do that thing.
We're gradually transitioning to renderware at Maxis, and so far I'm liking that a lot
better because it has this kind of toolkit concept, where basically you grab bits of
the code and use that directly.
So Charles said not to diss C++, but there are some areas of C++ that you just have to.
So our math library was written using template metaprogramming, which was all the rage a
few years back.
This was kind of a common problem on the Sims 2 engineering-wise, which is if people implement
new approaches, then they should always profile them and show that they're actually better.
So unfortunately no one ever did this, and it turns out that even with this template
metaprogramming idea, which is that you actually wind up with more efficient code because you're
throwing all of the math instructions at the compiler to do with what it will, we got a
decrease in performance over just a normal standard math library because we're basically
blowing the compiler's mind.
Any compiler is going to give up after a certain amount of inline depth.
That actually wasn't the problem.
The problem with this approach, which kind of leads to one of these non-obvious things
you have to be careful about, is its impact on debug speed.
So this approach to math vector library multiplied by a lot the number of function calls in any
particular math operation.
So if I remember right, a vec4 addition wound up being 20 function calls in debug.
That's with asserts off.
With asserts on, it was more like 40 or 50.
So when we did kind of a reverse patch job and tried to mitigate some of this stuff,
we found that the overall impact on debug speed was something like 75% on our core engine
routines.
So what this is pointing to is just you really have to be careful in the early stages of
your project in terms of having people who are experienced and practical because the
stuff they do will impact the later programmers immensely.
Another problem was just API churn.
So because we wound up with these kind of ghettoized, well not ghettoized, but separated
engineering teams, one of which was highly dependent on the other, but weren't really
talking to each other, you have this problem of API churn, where there was a number of
low-level APIs that were being changed constantly to meet some measure of conceptual purity
on the engine side.
That led to, you know, the obvious problems you might imagine.
So adding this all up, my point of view basically came down to a lack of productivity.
At some point you have to ship the game and you have to be careful that you don't attempt
too much kind of research-y code generation before doing that.
And one of our learning experiences is that we, by kind of taking the conceptually pure
approach and trying to do things right, we actually lost a lot of opportunities.
So the problem with taking this kind of approach is that you run out of time to basically finish
off stuff, or even get anything working.
So for instance, we have a lot of well-authored normal maps for our sims, but due to the slowness
of the whole normal map process, in the last couple of weeks of the project, the decision
was made to basically make them only ever show up on a 128-meg cart, because no one
ever actually implemented, you know, any kind of normal map compression, or even enough
code to be able to drop the level of detail of the normal map.
I've been through the static LODs.
We dropped our main shader path also in the last couple of weeks, for similar reasons.
It's just, you have a tight deadline.
You've got to keep close to the middle when you're making a lot of these decisions, or
you just will run out of time.
So some of that stuff is what I'm referring to when I say we shipped, because it kind
of amazes me that we did sometimes.
So that's lessons learned, you know, how we actually, what are we going to change in what
we're doing.
Like I said, one of our biggest problems was design churn, so what we're doing is learning
about pre-production.
This is something that other areas of EA have been reasonably big on, but we never have
it in Maxis, just because we haven't been in that arena until, say, Sims 2, or maybe
The Sims Online, where you have massive teams, and basically getting everything through the
sausage factory and out the other side is, you really have to do it in order.
So the whole purpose of pre-production is that you do all of your research then, all
of your prototyping, make as many design decisions as possible that are going to stick, solely
because every one of those that you do, you're reducing the risk later on.
And then, you know, theoretically you go through the slow ramp up to full production, slowly
throw more engineers at it, and it all works marvelously.
So we'll see how that goes.
For a project your size, how many people do you think would be on pre-production?
10 to 20.
So why didn't we do this before?
Because, after all, the early Sims 2 research team was about 5'10", 20'10", it kind of bounced
around depending on what other project was going on at the time.
Basically we just didn't have the right mindset.
We treated it as a research team, and most of that research went into graphics research,
or very limited new idea research that wasn't actually implemented and tested.
So the key is really, don't just make design decisions, get in there and prototype them,
and do some gameplay prototypes and actually play through and see whether it's going to
work.
In terms of some of our communication issues, you know, these issues are not due to anything
particular at Maxis, it's just the problem of having teams that large and getting communication
from one place to another.
We really kind of have to change the way that we work.
So we're trying this kind of pod style of working where we move much more to a kind
of a SWAT team approach that we have kind of used in the past.
So the goal is to get into these kind of small groups where the iteration turnaround time
is really fast and there's not a lot of kind of cross process overhead to slow people down.
This is a small example of that kind of thing that happened on The Sims 2.
So for the original ship deadline of January 2004, we actually cut swimming pools, because
we had none of it done, and it was going to be quite a bit of work, which was almost unbelievable.
Swimming pools are an essential part of The Sims game, because they're how you kill Sims.
I mean, you throw them in the pool, you take away the ladder, and they die.
So we got a skunkworks team together to help save them, driven by a producer, all completely
outside the normal process rules.
Simplest things we could do, basically just get something in there and get it working,
you know, a small number of people, and I think we did a really good job on kind of
getting that all in there and getting it working.
It was kind of a lesson in terms of how much kind of stepping outside the big meat grinder
process and just having a small SWAT team working on a particular feature could work.
And of course we added more stuff to that once the slip came down.
So we're already doing a lot of art pre-production on next-gen titles and other titles.
We even have concept artists being hired to do sketches to try and figure out, you know,
what our look will be and basically just explore, you know, the space of looks.
It turns out to be much faster to have an artist do that than to do a whole bunch of
Maya renders, or even worse, you know, get a game engine up and running and do things
like that.
A big success from Sims 2 art-wise was probably the auto-content build.
Once that was up and running, it worked very smoothly, and given the sheer amount of models
and animations that went through that system, I think it did a good job of getting everything
in the right place at the right time.
Engineering, basically the lesson is just keep it simple.
We way overdid the complexity on Sims 2, and that was pretty, actually pretty much localized
to that team.
I've had experience on other teams at Maxis, and they've been much more practical and much
more focused.
So basically, the simpler you can keep the base-level things, the more complexity you
can build on top of that.
I'm sure I'm not telling anyone anything particularly new here.
And basically, yeah, rapid iteration is, as some of the previous speakers have said, is
absolutely the key to game development.
Technology-wise, we're kind of moving to this commodity rendering system.
As someone said to me, the great thing about it is that it's something that everyone can
hate, kind of together, external.
We're adopting the effect system that kind of worked out pretty well for authoring large
amounts of dynamic content and complicated content, and also as a prototyping tool.
We're using that quite a bit on current pre-production systems to kind of figure out what something
will look like.
We're switching much more to text-based scripting, as I said before, and we had a kind of game
object asset database on Sims 2, but it was kind of, it was a carryover from Sims 1, and
it wasn't the best.
So we actually have a team working on building, you know, a proper kind of production hardware
system.
And that's pretty much it.
These are some of the people at Maxis who were helpful in giving me advice and statistics
and all the rest of it.
All right.
Thank you.
Thank you.
So, who's got questions?
All right.
Go ahead and start, and I will bring the mic.
So, I have a couple of questions for you to answer.
Why was your content building system centralised?
Why was it important for your artists to build their content on the server rather than for
them to actually be building the content on their machine?
Like, why does an artist need to check something in in order to validate it?
That was something I didn't quite understand.
They could validate it locally.
So this is just art that is actually going into the game, you know, into the next build,
daily build.
Locally they would be developing this model animation that had a preview button, and it
would kind of do an instant export then and fire up the remote, the, sorry, local viewer.
We had a very lightweight kind of viewer application, and basically I'd be able to
preview it in there.
That was another sign of actually putting it in the game for a re-build.
Right.
You don't need to put it in the game for a re-build if it went through the third person
and you put it in the middle.
Yes.
The content building is the same.
Yeah.
Same code.
At the beginning, there was a lot of talk about the importance of the tech lead, and
kind of vetting decisions.
I'm wondering how your, this comm system got built, and your templates got built, and all
these, how these things lasted more than a week or two in development.
Yeah.
I have to be somewhat tactful here.
You've got to have good tech leads.
That's what it comes down to.
These weren't things that were pushed on you by, like, I noticed the outsourcing was
pushed on you.
Were these other things pushed on you by EA or other people?
Right.
Right.
That's not the answer.
Come on.
You're right.
So they shouldn't have lasted more than a few weeks.
I mean, there were obviously bad things at the time.
Unfortunately, the people who pushed them basically had the support of upper-level engineering
management who weren't really used to, had not had engineering experience before.
So you talked about the third-direct scheme for optimizing graphics.
How much do you think the scene graph was a factor in having to go to that scheme?
Well, you see, it was, I think it was a large factor.
It's not the only factor.
We would have had to do something else on low-end machines, because we were definitely
up against the batch count.
But there are a number of other things we could have done there, you know, accumulating
objects, well, A, better LOD, and B, accumulating objects into multiple buffers, and all sorts
of things.
This question may not be relevant, so please feel free not to answer it, but you mentioned
twice that the toolkit approach, you know, you thought was something really good you
wanted.
What exactly is that?
I mean, rather than providing, like, a top-level interface that dictates how something is going
to be rendered, which was pretty much what we had, you instead get a bunch of utility
routines that you're assembling into, you know, a rendering engine.
So one of our problems was that the interface between, you know, our graphics layer and
our gameplay layer was far too high.
So there were a lot of things that, say, we wanted to do in the gameplay arena that
we couldn't, without actually reaching down to the graphics layer and getting changes
made.
And by the same token, which meant we were kind of limited in what we could do.
I had one other thought, but I forgot that.
After the slip, was there expectation to up the target that you were going for, rather
than a DX7 to a DX8 kind of platform, because you were slipping by about seven, eight months?
Yeah.
No.
And I think the reason for that is that we were initially only slipping five or six months,
and then we slipped again.
But it's not a slip if no one outside hears about it.
Yeah, but the pressure wasn't actually on at that point, because at the time where we
could actually have done anything about this, and I'll point out, we did ship with a DX9
path.
The reason that we cut the DX8 path was our system was too inefficient to handle the massive,
you need a lot of different shaders to be generated under that kind of path, as Chris
was talking about earlier, and that killed us.
But at the time, there weren't a lot of titles out there that were even using DX8 cards well
on the PC.
Was there some guy whose entire job was to do the DX8 path, and then it got cut, or anything
like that?
It wasn't his entire job.
We did actually, towards the end, say for the last year, we had a guy who was pretty
much dedicated to material scripting.
Up to that point, it had been done by me and another guy.
So yeah, that probably was the majority of his job, because all that stuff had to be
written in Asm, and a lot of different cases to handle all the different, you know, to
pack stuff in.
An interesting stat is that the object we had with the largest number of bones was actually
the bed, which had 180 bones, and obviously had to be split up for the hardware.
That's just because there are a lot of different ways to get into a bed, and you know, the
blankets have to ruffle up, and all the rest of it.
It's never the objects that you kind of expect that are the most complicated.
Okay, so I'm going to do Ken, and then we're going to be open now to both further questions,
and also just if you have statements, or you want to say anything about your own development
process, we'll kind of pass the mic around.
So I wanted to ask you about the structure of the engineering team, and how hierarchy
worked.
You know, was there someone who sort of ran herd over scheduling, in addition to, you
know, perhaps multiple programming leads?
You know, how did that all sort of flesh out?
So one of our problems was that it changed a lot.
That was probably actually a problem I didn't touch on on the Simstude, like right throughout
the process.
I mean, certainly up until the last couple of years, we were constantly changing producer,
or whoever was running the team, as they got sucked off to, you know, be thrown on the
latest fire on other projects.
So for the last couple of years, we had a development director, and the engineering
staffing under that, she was directly running the engineering team, like most of the engineering
team, and then the leads were kind of over here.
And I think one of the things that kind of went wrong is that the leads had a bit too
much autonomy, and a bit too much separation from the rest of the engineering team.
So you think, in your perfect world, would that not happen again?
Yeah, in my perfect world, we would have a dedicated engineering manager, and everyone
would be below that.
How many leads?
I mean, a lot of it just depends on the people you have involved.
Like the previous title, SimCity 4, we had basically one guy who was pretty much the
tech lead, and he was highly competent, and everything pretty much just worked.
How many leads were there?
On Sims 2?
Yeah.
It depends how you classify it.
There was a graphics lead.
Am I just asking the wrong question here?
There was a graphics lead, there was an animation lead, there was kind of a lead engineer who
covered everything else, but tended to ping-pong into management and back as we went through,
like, headcount changes.
And so, you know, didn't really get heavily involved.
What if I used you as a poet, maybe?
I'm confused.
I don't know.
We do have a whiteboard, if you want to do that.
We understand if you don't, but, you know.
So, you're after, like, the engineering structure?
Yeah.
Yeah.
Okay, so towards the end...
It's longer than time, towards the end, right?
Okay, so moment one, towards the end.
It was basically DD, graphics, animation, and, like, other lead.
Development director.
Yeah, so another problem we had then was that our development director, because we were
doing a lot of this massive CM stuff for the first time, a lot of the attention was kind
of diverted onto CM and managing the daily build.
So there wasn't always a lot of attention left over for the engineering process.
So DD leads...
I mean, the problem here is just that it was confused.
Like, I can draw a picture at one stage which basically had most of the engineers under
this third lead.
Where are we?
Graphics, animation, simulation.
Um...
So, like, this was just before the slip, I think.
After the slip, you know, it was obvious that DD was overburdened, so we had more, like,
three engineering managers who came in and basically each took a bit of the team down
here.
And then unfortunately we lost one of them six months later.
Um...
Did people sort of migrate over to the graphics and animation leads?
Did those people actually manage, or were they really just sort of pioneering their
own?
So...
Um...
So there's manage and then there's task assignment.
Yeah.
So basically...
That's the thing I'm asking about.
Okay, so task assignment was basically these guys talking to the DD in some cases, let's
say specific graphics tasks.
So normal mapping.
You know, with, say, this guy talking to the DD and then assigning tasks to junior programmers.
With, say, this guy then asking to see, like, a specification of an interface or things
like that.
Animation was very nicely compartmentalized, actually.
Basically, the reason the animation lead didn't have a lot of direct reports is that he managed
to maintain that system pretty much all the way through in close conjunction with, you
know, the technical art director.
And only at the end had to work with another engineer to implement animation compression
and all sorts of things like that.
So I think a lot of people under the simulation guy, they were really being tapped out by
the development director?
Yes.
Okay.
And so...
For a while.
Who...
The answer to this question is not applicable, but, you know, who enforced policy standards
across the team?
Code policy standards or...
Yeah, policy standards.
Like, you will deallocate your freaking memory allocation when you clean up your...
Initially, when the team was smaller, the graphics lead and the simulation lead did.
As the team grew larger and it became obvious that some of these things were bad ideas,
everything became much more freeform.
I'm not sure I can kind of get into too much detail.
Yeah, back here.
So while I'm waiting for the microphone on top...
Go ahead.
So this is sort of a tangent, but something we're trying is a thing called team software process.
Carnegie Mellon Software Engineering Institute does it.
I don't know if anyone's heard of it, but we ran into a project.
I mean, it wasn't nearly as big as Sims, but where it was kind of crazy and out of control.
And after that, we wanted to figure out, you know, how could we make it better?
And so we tried this thing called team software process where sort of all the team members
are responsible for coming up with the schedule themselves.
And they're responsible for tracking their own time.
And there's sort of time set up for reviews and everything.
And there's a lot of cross-team communication, which seemed to help with a lot of these sort of areas
where you don't know who's responsible for what.
And you don't know if the communication is going across the team or back up or stuff like that.
I don't know.
So part of the problem here is that this is kind of a chicken and the egg situation.
There was a lot of churn here just because it was such a high pressure product.
And as a result, we had a lot of change.
You know, if something wasn't working, it would be changed.
We lost people too.
It might do me better to kind of diagram the situation on SimCity,
which had almost as many engineers.
We had like 22 engineers in SimCity at the end, as opposed to about 28 on Sims 2.
Where we just had a very competent kind of...
We called them a DD, but at that point it was like the Maxis DD,
which is kind of more of an engineering manager.
We had a tech lead.
And then basically all the engineers were being managed by the engineering manager
and being given a large amount of technical direction by the tech lead.
But not too much.
I think one of the problems with Sims 2 was actually there was too much technical direction.
There was too much.
Yeah, too much processes is what I'm trying to say.
Yeah, just to follow up with what we're doing,
it's something similar to what you described, and it's called Scrum.
And it's more of an agile methodology where a team's self-organized.
They break down their own tasks.
They create their own estimates.
And then they essentially meet on a daily basis to re-estimate remaining tasks
or basically burn down on this monthly iteration that they do.
And it does remove a lot of burden from lead programmers, managers,
for having to break down these tasks and estimate them for other people.
Right.
Yeah, so the blob wasn't quite as ill-defined as I'm making out here.
You're reminding me.
You know, we had teams within that who their function was well understood.
You know, we had some people working on a particular section of,
say, the dirty rec system.
Others on effects and lighting.
Others on the simulation routing.
I'm just unable to give a very clear answer to, like, the overall structure.
Just because it changed recently, often.
Two kind of silly questions, I guess.
Feel free to ignore them.
Like, one, could you quantify why the product slipped at one level
of more granularity?
Like, it was design.
It was code.
The scene graph screwed us.
I mean, was there, like, is there any way to drill down that one more level
to kind of, like, place more blame on the slip?
During the talk, it was pretty much 50% design, 50% engineering.
The engineering didn't get as much play because it was so obvious.
You know, the executive producers were mostly concerned with the design.
They all had input into that.
But the fact was also we couldn't have made engineering by a considerable amount.
The second kind of silly question is, like, what's the feedback loop here,
like, corporate-wise?
I mean, the game came out and beat Forecast.
It did slip, which cost money.
But, like, you know, kind of screwed up project, yet,
you know, we're still making it.
So, like, where's the, like, where do we, when does,
and this is kind of a meta question, like, when does process
actually end up screwing us?
Like, in this case, it costs EA, you know, 20 million more dollars,
but the game's still going to make $700 million or something insane like that.
So, like.
Right.
I actually had a bullet point on one of my slides at some point
kind of saying, don't take the wrong listen from this.
Because one of the listens is, too, is kind of,
you can screw up a project, but you can't screw up a game.
So, like, when does, and this is kind of a meta question,
like, when does process actually end up screwing us?
Because one of the listens is, too, is kind of,
you can screw up and survive.
And I don't think we can screw up to this extent
and continue to survive.
So, that's why I'm also saying we have to improve our process.
Yeah, I was just going to say, one of the things you mentioned,
and there was a question over here about when you're slipping,
did they make you think of other graphics cards?
And I know every publisher I've ever dealt with,
they say, you can slip, but you've got to get me
my seven pet peeve features that you weren't going to give me before.
And that always ends up almost costing me more than the slips.
So, we were aware of that problem, yeah.
And we were, at least on the engineering side,
very careful not to add, basically,
every new engineering feature that was going to be added
during the slip had to go through a pretty formal process,
like a natural TDD and all the rest of it.
Yeah, because otherwise you could just get through the month.
Yes, yes.
There's also that infamous mythical man month thing about,
if you're going to slip, you know, slip a long time.
Don't do incremental slips and just...
You know, the other bad thing about slipping is just,
you've had people who were crunching for, say,
four or five months, doing six or seven day weeks,
and extending that for another six months.
So, I was just going to comment on something Chris said
about the, like, when is the process thing actually going to hurt,
which is just that I think that it's pretty clear
that a lot of times the process thing,
when things slip long and that sort of stuff,
it doesn't actually hurt because there's not, like,
sufficient competition in the games, you know,
in the games that are out there to make that actually hurt you.
You know, I mean, if a game like The Sims 2 slips for a year
or two years, it doesn't really matter
because there aren't, like, competitors coming in
and selling to that market aggressively
in a way that takes away from it.
I mean, that's usually when companies fail
because of process things.
It's because someone comes and, you know,
takes advantage of that.
So, I would actually slightly disagree with that.
As the games get bigger and bigger,
the cost increases for, like, a number of reasons.
One is just marketing.
So, we lost a lot of money by not hitting that date
just because we already had, like, TV ads booked,
you know, various things like that.
The other is burn rate.
You know, the burn rate for slipping two years
is phenomenal with a big project like that.
I was going to ask a question
which might lead to a more general discussion,
which is, can you talk about the background
that your object engineers had?
Like, you refer to them as object engineers,
which I guess means that they are trained in engineering,
but at the same time,
they're exclusively using a visual tool
and getting frustrated with it
because they'd rather be using command line text-based things.
So, kind of, what is their background?
Were they engineers and designers?
Would you rather have had engineers?
Would you rather have had designers?
And how would you change the tools
just in a general sense to tailor to that?
So, they weren't, you know,
they weren't C++ engineers.
They were, although some of them,
you know, had that capability,
they were gameplay scripters.
They had to basically,
the way I tend to think about them is just,
you know, I've had some experience
in the animation industry
and that kind of, that TD-level glue
that holds the whole process together.
So, you know, they're taking the assets,
you know, there's the engine
that's being supported,
the engineers,
and together with the producers
knitting it all together
to make the game out of it.
In a game like The Sims,
where, you know,
the gameplay is nicely compartmentalized like that,
you know, the kind of combination
level designer,
yeah.
Yeah, basically.
All right.
Well, thank you, Andrew.
Thank you.
Thank you.
Thank you.

transcript.vtt

WEBVTT

00:00.001 --> 00:08.000
All right, we've got one more game-oriented lecture coming up, and it's given by Andrew

00:08.000 --> 00:13.160
Wilmot of Maxis about The Sims 2, which should provide a pretty interesting contrast to the

00:13.160 --> 00:17.440
previous games, which were all shooters about shooting things and stuff.

00:17.440 --> 00:19.680
So go for it.

00:19.680 --> 00:28.960
Yeah, this was quite a different game.

00:28.960 --> 00:32.520
And it's also by far the most content-heavy game I've ever worked on.

00:32.520 --> 00:37.720
So what I'm going to do is mainly give kind of a breadth-first overview of the entire

00:37.720 --> 00:41.800
process, and just try and hit as many different parts of the puzzle as possible, rather than

00:41.800 --> 00:45.160
drilling down deep on anything too much in particular.

00:45.160 --> 00:50.400
So in questions at the end, I'll be happy to go into detail on some stuff, but the main

00:50.400 --> 00:54.480
theme of this talk is just to try and give an overview of how you put together what was

00:54.480 --> 00:58.920
really a pretty massive game.

00:58.920 --> 01:01.600
So yeah, why is Sims 2 interesting?

01:01.600 --> 01:03.800
A lot of animation.

01:03.800 --> 01:07.480
Like some of the other talks here, a lot of sound, surprisingly.

01:07.480 --> 01:09.720
Massive amounts of people.

01:09.720 --> 01:14.600
We've had at least probably 250, maybe 300 people touch this project.

01:14.600 --> 01:18.440
That's not all at once, but still.

01:18.440 --> 01:22.400
It's interesting from the point of view that the users are our level designers.

01:22.400 --> 01:27.120
You know, this is completely different from the normal FPS process, where you have pretty

01:27.120 --> 01:31.480
much the whole point of your content is to produce levels.

01:31.480 --> 01:36.740
So our users are designing the levels, we're just giving them the content.

01:36.740 --> 01:40.000
And our simulator is kind of interesting in that the whole thing is driven by a visual

01:40.000 --> 01:43.720
scripting language.

01:43.720 --> 01:45.840
So the Sims 2 was a long project.

01:45.840 --> 01:48.760
We started in late 2000.

01:48.760 --> 01:53.160
For a long time, it was just a pretty small research team basically doing constant demos

01:53.360 --> 01:57.840
and constantly losing people or gaining people from the rest of the studio due to other projects

01:57.840 --> 01:59.520
going on.

01:59.520 --> 02:01.560
We were in full production for about two or three years.

02:01.560 --> 02:04.160
A lot of people at the end, as I said.

02:04.160 --> 02:06.960
And we had a pretty big slip.

02:06.960 --> 02:11.280
We were trying very hard to hit a particular deadline, and we missed it.

02:11.280 --> 02:13.600
So there's also some interesting stuff there.

02:13.600 --> 02:17.200
The downside of that for us was that we had an extended crunch period.

02:18.200 --> 02:23.800
I'm going to give more stats as we go along through each particular section, but these

02:23.800 --> 02:25.520
are like the main ones.

02:25.520 --> 02:30.680
For me, the 11,000 shipped animations is the really big one.

02:30.680 --> 02:36.960
A lot of the other kinds of data go into a game, but the 1.1 million lines of code quoted

02:36.960 --> 02:43.360
there is no blank lines and no comments.

02:43.360 --> 02:46.560
I'm just going to quickly flip through a couple of diagrams, which you're not expected to

02:46.560 --> 02:51.920
memorize, just roughly showing what the process was on Sims 2, so that as I go through the

02:51.920 --> 02:56.200
sections a bit later, you have something to hang your thoughts on.

02:56.200 --> 03:02.960
This is basically how our art and object scripter process kind of went through the pipeline

03:02.960 --> 03:11.720
and produced mostly either scripts or 3D or 2D resources.

03:11.840 --> 03:17.680
On the other side, we had code picking up these resources, and our architecture was

03:17.680 --> 03:24.400
basically that we had kind of a platform, a game-neutral rendering engine, and then

03:24.400 --> 03:29.560
a diverse bunch of components basically making up the application layer, mostly driven just

03:29.560 --> 03:35.480
by object scripts, text scripts, and binary resources.

03:35.480 --> 03:41.600
A lot is like a saved house, a saved file.

03:41.720 --> 03:48.000
I'm going to run through a few of the more interesting areas of the Sims 2 process, and

03:48.000 --> 03:51.720
at the end I'm going to look at the lessons we learned and how we're moving forward with

03:51.720 --> 03:53.960
some of these lessons.

03:53.960 --> 03:55.400
So we're starting with art.

03:55.400 --> 04:03.640
In many ways, a lot of what we try and deliver is basically the output of our animators as

04:03.640 --> 04:05.040
entertainment to our users.

04:05.040 --> 04:08.800
I mean, obviously that's not everything, but it's a significant fraction of it.

04:08.800 --> 04:14.600
So you can kind of view a lot of our pipeline as a kind of content delivery system.

04:14.600 --> 04:18.720
So these are some stats from art.

04:18.720 --> 04:21.520
A lot of models as well as animations.

04:21.520 --> 04:27.000
A lot of effects, which we'll get to a bit later.

04:27.000 --> 04:31.000
Basically just a lot of data.

04:31.000 --> 04:34.520
This is somewhat less interesting UI.

04:34.520 --> 04:38.000
I'm actually not going to talk about UI too much because it ran pretty smoothly.

04:38.000 --> 04:43.600
It was, we basically have UI down as a problem at this stage, which contrasts quite heavily

04:43.600 --> 04:47.800
to some of the other parts of the game.

04:47.800 --> 04:51.280
So we had a lot of artists.

04:51.280 --> 04:57.560
Our technical art director likes to tell the story that, he comes from Pixar and he worked

04:57.560 --> 05:01.880
on the first Toy Story movie, and the first Toy Story movie actually had fewer animators

05:01.880 --> 05:03.760
than we did on this project.

05:03.760 --> 05:09.160
So that's kind of giving you an idea of where games are getting compared to earlier computer

05:09.160 --> 05:13.440
animation.

05:13.440 --> 05:15.000
Our art tools were pretty vanilla.

05:15.000 --> 05:23.440
So Maxis had been kind of a Maxis-based studio before this title, and we, Sims 2 was really

05:23.440 --> 05:27.520
the start of the whole Maya transition, and now we're mostly Maya-based.

05:27.520 --> 05:32.640
And a lot of that is just because we find it a better tool for animation.

05:32.640 --> 05:37.440
Basically we bought over all of the animations from Sims 1, which were in Max format, wrote

05:37.440 --> 05:43.960
a complicated pipeline to kind of drive re-rigged animation in Maya by the source animations

05:43.960 --> 05:46.760
imported from Max.

05:46.760 --> 05:50.760
We didn't use any of this stuff in the end, but it was pretty useful as kind of like an

05:50.760 --> 05:51.760
initial guide.

05:51.760 --> 05:55.640
The reason we didn't use much of it is just that some of the animations were no longer

05:55.640 --> 06:01.200
relevant and everything that was basically got redone to a much higher kind of level

06:01.200 --> 06:03.080
of detail.

06:03.080 --> 06:06.960
Like some of the other previous talkers, we used Null a lot for basically all of our UI

06:06.960 --> 06:14.440
within Maya, and it did a very good job of that.

06:14.440 --> 06:20.000
Our pipeline, again, is reasonably straightforward.

06:20.000 --> 06:25.440
One interesting thing we did is about halfway through the project, added direct read support

06:25.440 --> 06:30.320
for Photoshop, and what this allowed us to do is have the artists work on all of the

06:30.320 --> 06:34.560
layers for, say, a sim or a model in a single Photoshop file, which helped their process

06:34.560 --> 06:38.160
a lot.

06:38.160 --> 06:43.820
In terms of animation and modeling, basically we went from Maya, we used a EA internal kind

06:43.820 --> 06:50.080
of platform neutral export format, and then we had our own asset compiler called Go Disco

06:50.080 --> 06:53.640
that converted that into game resources.

06:53.640 --> 06:56.920
Now the way this all worked was kind of interesting.

06:56.920 --> 07:00.880
The artists basically checked directly into Perforce, and the reason they didn't forget

07:00.880 --> 07:05.800
to check into Perforce is that was basically, there were two parts to that step.

07:05.800 --> 07:10.960
You checked in your new animation or model or whatever, and then that would trigger the

07:10.960 --> 07:16.520
build machine that would then convert that content to the game format, and you'd get

07:16.520 --> 07:21.760
an email back telling you if there were any errors or whether the process had gone smoothly.

07:21.760 --> 07:27.640
So you weren't done until you got that email back and it was all clear.

07:27.640 --> 07:34.060
This whole process also built a big web page with basically a link for every single asset

07:34.060 --> 07:35.060
in the game.

07:35.060 --> 07:40.160
So you could browse this website and drill down to any particular model and all of the

07:40.160 --> 07:46.840
game events that might be in it, or number of vertices, you know, you name it.

07:46.840 --> 07:50.560
The Sims 2 skeleton increased in complexity quite a bit over Sims 1X.

07:50.560 --> 07:57.000
Sims 1X actually had smooth skinning, which I was initially surprised to find out about.

07:57.000 --> 08:01.480
A lot of the extra attention to detail went into the facial morph targets, is where we

08:01.480 --> 08:07.480
got a lot of our extra kind of emotion and expressiveness, which is a bit dark so you

08:07.480 --> 08:12.880
can actually see it by the game, I guess.

08:12.880 --> 08:16.800
So the main skeleton had 64 weighted bones that were actually being processed by either

08:16.800 --> 08:22.080
the GPU or on CPU, but the skeleton itself had 116.

08:22.080 --> 08:26.600
So a lot of these extra bones were for grips, you know, where to put your spectacles, where

08:26.600 --> 08:32.760
to put your hat grip for matching up IK, and all sorts of things like that.

08:32.760 --> 08:38.080
And it was kind of because of all those extra slots and so on that we didn't lock the skeleton

08:38.080 --> 08:39.080
in the end.

08:39.080 --> 08:44.440
So this was a big focus of ours early on with Sims 2, it was like, you know, finalize the

08:44.440 --> 08:48.200
skeleton, get the number of bones and the layout and everything absolutely rigid, and

08:48.200 --> 08:51.160
then we'll lock it and forget about it.

08:51.160 --> 08:55.920
But that basically didn't work because as design evolved, we needed to add, you know,

08:55.920 --> 08:58.880
grips, remove grips, modify our skeleton.

08:58.880 --> 09:04.280
So instead we had a system whereby all of our animations referenced one master Maya

09:04.280 --> 09:06.940
file that contained the skeleton.

09:06.940 --> 09:11.640
And we had an indexing system that made it reasonably easy to kind of add bones or even

09:11.640 --> 09:20.640
remove them without invalidating all the animations.

09:20.640 --> 09:27.280
So modeling, we're actually lucky enough to have a lot of the Sims 1 modeling modelers

09:27.280 --> 09:31.280
come through, which is kind of important to the Sims because, you know, they developed

09:31.280 --> 09:35.400
a particular look in Sims 1 that we were basically trying to amp up, and they also had a lot

09:35.400 --> 09:44.320
of experience and ways to handle texturing and modeling on an animated character when

09:44.320 --> 09:48.000
you have, you know, hundreds of skins maybe.

09:48.000 --> 09:52.880
There's a certain amount of process there that has to run more smoothly than with a

09:52.880 --> 09:53.880
fewer number.

09:53.880 --> 10:00.640
One of the things that went wrong was that the graphics team didn't really give artists

10:00.640 --> 10:03.840
either a poly budget or a texture budget.

10:03.840 --> 10:08.200
This was very kind of hand-wavy, you know, that the code will take care of it, will either

10:08.200 --> 10:11.200
write dynamic simplification.

10:11.200 --> 10:17.680
What we actually shipped, did manage to ship with respect to the textures was that basically

10:17.680 --> 10:21.960
we dropped mid-levels from all of our textures that are currently being referenced until

10:21.960 --> 10:23.960
it all fits within video RAM.

10:23.960 --> 10:28.600
But that's a pretty kind of clunky and error-prone process, so I don't think it really worked

10:28.600 --> 10:30.640
out.

10:30.640 --> 10:35.960
We contracted out our object LODs, which seems to be the thing to do these days.

10:35.960 --> 10:40.720
Didn't work as well as you might think because there's a lot of overhead in contracting out

10:40.720 --> 10:44.880
our, in terms of getting the external team up to speed on your process.

10:44.880 --> 10:48.240
And if you don't get that exactly right, you wind up having to fix every animation or model

10:48.240 --> 10:52.160
that comes back, and we had a fair bit of that.

10:52.160 --> 10:57.160
So because we had a lot of models, we implemented a bunch of different visualization modes to

10:57.160 --> 11:00.600
try and spot kind of bad model usage.

11:00.600 --> 11:03.800
So we had textual density visualization, so we could try and figure out where we were

11:03.800 --> 11:06.320
wasting texture memory.

11:06.320 --> 11:14.120
We had a neat little animated model display where basically the content viewer that we

11:14.120 --> 11:19.160
used to view our models or animations would repeatedly lurk between, say, a model and

11:19.160 --> 11:24.640
then its layout in the texture map, so you could kind of see at a glance where the texture

11:24.640 --> 11:28.920
map was going on that particular model.

11:28.920 --> 11:34.240
Obviously late in the game, we also had to implement this thing to let the artist switch

11:34.240 --> 11:37.160
between the different shader parts in the game.

11:37.160 --> 11:43.760
This was kind of because the shader parts were so buggy.

11:43.760 --> 11:50.040
We wanted to make sure that they were tuning the lighting and so on to the right thing.

11:50.040 --> 11:56.320
Speaking of lighting, this is one of the most well-processed parts of the game.

11:56.320 --> 12:03.480
The art team actually produced a quite extensive TDD on the whole lighting design, and we even

12:03.480 --> 12:08.280
prototyped it beforehand.

12:08.280 --> 12:12.200
We had two lighters during the course of the game, kind of one before slip and one after

12:12.200 --> 12:17.600
slip, and we pretty much had dedicated people for that because rather than tuning lighting

12:17.600 --> 12:23.200
for a level or something, they were tuning a system that had to try and light the game

12:23.200 --> 12:27.440
no matter what the user did, no matter where they placed their lamps or built a house or

12:27.440 --> 12:28.880
anything like that.

12:28.880 --> 12:35.880
So that was kind of a new challenge for us, and some of it worked, some of it didn't.

12:35.880 --> 12:40.800
Just quickly on what we did for level of detail, so this is a lesson in kind of talking about

12:40.800 --> 12:43.640
something a lot and not actually sitting down and implementing it.

12:43.640 --> 12:48.200
We wound up with static LOD, never formally spec'd.

12:48.200 --> 12:55.760
All of our sims had at least one LOD with fewer bones and about half the poly count.

12:55.760 --> 13:02.920
And in fact, we don't even switch LOD dynamically within a lot.

13:02.920 --> 13:04.600
I didn't quite know where to throw this in.

13:04.600 --> 13:06.360
This is sound.

13:06.360 --> 13:09.080
Maybe it's art, maybe it's engineering.

13:09.080 --> 13:12.840
We had a lot of sound, like some of the previous games.

13:12.840 --> 13:18.020
The stat that always gets me is 43,000 Vox samples.

13:18.020 --> 13:22.900
And the reason that's kind of mind-blowing is that all of our sims speak in simlish,

13:22.900 --> 13:27.780
so we don't even have to localize that.

13:27.780 --> 13:33.300
So if you imagine having to localize all those Vox samples, it would just be insane.

13:33.300 --> 13:38.020
Lots of footsteps and just a lot of ambient sound kind of triggered off the camera and

13:38.020 --> 13:41.200
triggered by various objects and so on.

13:41.200 --> 13:45.260
And sound was always, at one point sound was actually two-thirds of our data footprint,

13:45.660 --> 13:46.660
using various compression methods.

13:46.660 --> 13:49.980
We got that down to a gigabyte.

13:49.980 --> 13:52.740
But it's a big part of our game.

13:52.740 --> 13:55.500
Okay, design.

13:55.500 --> 13:58.660
This was like an early version of The Neighborhood that thankfully was replaced by something

13:58.660 --> 14:02.140
much prettier.

14:02.140 --> 14:06.140
Design was one of the biggest areas in Sims 2.

14:06.140 --> 14:07.980
It was really difficult, basically.

14:07.980 --> 14:13.660
You're trying to improve on Sims 1, which is one of the best-selling games of all time,

14:13.660 --> 14:18.540
and attract people who haven't played the game before, bring back people who played

14:18.540 --> 14:22.100
it for a while and got bored, and not lose all the people who are still obsessed about

14:22.100 --> 14:25.060
the original or buying all the expansion packs.

14:25.060 --> 14:31.060
Also, we have surveys showing that people play The Sims in very different ways.

14:31.060 --> 14:35.180
It's almost an even split between four different styles of play.

14:35.180 --> 14:39.020
I can never remember all of them, but it's something like there are people who play the

14:39.020 --> 14:44.620
romance game, you know, the actual simulated bit, people who just build houses, people

14:44.620 --> 14:52.020
who basically like to torture their Sims, that's a big one, and something else.

14:52.020 --> 14:57.500
So we set ourselves a goal about two years out from ship four to get a 90-plus Metacritic

14:57.500 --> 15:01.340
rating, because this was a big thing at EA at the time, and I think still is.

15:01.340 --> 15:05.620
If you can get that rating, then your sales go up by a certain factor of X.

15:05.740 --> 15:11.540
And we actually hit that, you know, we're still at 91, I think.

15:11.540 --> 15:18.780
So there's all of that, and then there's the pressure that comes from a high-profile title,

15:18.780 --> 15:27.540
which means a lot of constant demo pressure, so I don't think the Oddworld guys suffer

15:27.540 --> 15:33.460
too much worse than us in terms of, like, constant demos and having to deal with...

15:33.460 --> 15:38.220
So while we're not selling to external producers, we are selling to EA executives, and we have

15:38.220 --> 15:41.860
to keep them on board with our changes and so on.

15:41.860 --> 15:44.780
We have this constant question of, you know, we're throwing new stuff in there, trying

15:44.780 --> 15:47.460
to make it compelling, where do we stop?

15:47.460 --> 15:51.060
If we take too long, then we basically have to throw more stuff in, because it's already

15:51.060 --> 15:53.540
been too long since the last one.

15:53.540 --> 15:57.980
And early on, we also lost a lot of people and a lot of ideas to, well, the expansion

15:57.980 --> 16:02.780
of Simpson's 1X.

16:02.780 --> 16:04.780
So new gameplay.

16:04.780 --> 16:12.580
We started off with movies as the main idea, you know, let people do the machina video-making

16:12.580 --> 16:13.580
thing.

16:13.580 --> 16:14.580
Yeah.

16:14.580 --> 16:18.900
And we actually came back to that as almost the last idea.

16:18.900 --> 16:23.180
We brought on aging, the whole idea of having a sim grow up through its life cycles, having

16:23.180 --> 16:28.620
four different age brackets and doing transitions between them.

16:28.620 --> 16:31.940
And then we threw another thing onto the pile of generations and genetics, you know, family

16:31.940 --> 16:36.020
trees, breed, crossbreed your sims and see what they look like.

16:36.020 --> 16:37.740
Horrific, really.

16:37.740 --> 16:42.260
Then, big life moments that we were trying to bring in, you know, some of the oomph that

16:42.260 --> 16:48.580
other games get kind of out of, you know, you defeat the boss monster and you get some

16:48.580 --> 16:50.420
great cutscene.

16:50.420 --> 16:55.660
So we were trying to emphasize some big moments in the gameplay.

16:55.660 --> 17:00.780
Aspirations and then wants and fears was pretty much the last one and that's probably, I think,

17:00.780 --> 17:03.100
the most compelling new bit of gameplay.

17:03.100 --> 17:09.520
So we kind of drew the line at that.

17:09.520 --> 17:13.900
So wants and fears basically and aspirations I think came after the slip.

17:13.900 --> 17:17.500
So once we'd slipped and half of the reason we slipped was because we didn't feel the

17:17.500 --> 17:20.380
design was there.

17:20.380 --> 17:24.780
We needed extremely tight process control to make sure that we didn't slip again.

17:24.780 --> 17:28.260
So we had very strict kind of change review.

17:28.260 --> 17:32.060
Anything that was going to be changed about the game had to go through meetings.

17:32.060 --> 17:37.540
We dedicated people to particular features and we basically had SWAT teams to try and

17:37.540 --> 17:44.180
bring these new features, design additions to completion without screwing up the currently

17:44.180 --> 17:47.780
working part of the game.

17:47.780 --> 17:50.420
So in the end it shipped.

17:50.420 --> 17:52.780
That first point is kind of redundant.

17:52.780 --> 17:54.820
We just could not have shipped at the original time.

17:54.820 --> 18:00.140
We probably couldn't even have put anything into a box.

18:00.140 --> 18:06.540
We got enough crucial extra gameplay, the wants and fears thing I think was crucial,

18:06.540 --> 18:10.540
and we also got the chance to finish engineering which was the other reason that we slipped.

18:10.540 --> 18:12.660
We were way behind on engineering.

18:12.660 --> 18:16.500
And in the end it was quite a success.

18:16.500 --> 18:21.700
So the lesson from that if you kind of go back at Maxis a year is never give up because

18:21.700 --> 18:26.460
things were looking pretty bleak at one stage.

18:26.460 --> 18:30.740
There's always a cost when you slip.

18:30.740 --> 18:37.020
So basically having to constantly redesign and we'll revisit this is always a negative.

18:37.020 --> 18:41.580
Design changes are really costly and all of the post mortems I read about this project

18:41.660 --> 18:46.460
were basically talking about how difficult it was to deal with constant change.

18:46.460 --> 18:51.900
We also got sucked into the EA Redwood Shores mothership early 2004 after the slip.

18:51.900 --> 18:55.380
I think they wanted to keep a closer eye on us.

18:55.380 --> 18:57.900
And there were various kind of negatives that came out of that.

18:57.900 --> 19:05.620
So we have a very big incentive to kind of learn how to do this kind of thing better.

19:05.620 --> 19:08.020
So object engineering.

19:08.060 --> 19:10.140
So this is probably as important as art.

19:10.140 --> 19:13.820
These are the guys who basically do all of the gameplay in The Sims.

19:13.820 --> 19:18.420
So the way The Sims works is that there's a simulator running a bunch of objects, you

19:18.420 --> 19:23.740
know, objects on various tiles and each object has a whole series of associated scripts and

19:23.740 --> 19:35.580
is also responsible for sequencing all of the animations that involve that object.

19:35.580 --> 19:38.980
So mostly this continued to work out well.

19:38.980 --> 19:42.540
I mean, the simulator is a reasonably nice process.

19:42.540 --> 19:46.500
We basically have it done in terms of adding objects and so on, you know.

19:46.500 --> 19:49.660
We've built up a lot of experience from the expansion packs and so on.

19:49.660 --> 19:52.060
There's still some problems with it.

19:52.060 --> 19:58.340
The scripting language is pretty simplistic and often what we're trying to expose to it

19:58.340 --> 20:00.900
from the game is reasonably complex.

20:00.900 --> 20:07.060
So you wind up with some pretty hairy scripts, which is what I'm referring to in that mismatched

20:07.060 --> 20:11.380
line.

20:11.380 --> 20:14.900
Another thing that goes on is this is the way we do our blending, as I mentioned before.

20:14.900 --> 20:19.460
So the artists basically provide all the source animations and then it's the object engineer's

20:19.460 --> 20:24.140
role to kind of blend those together to produce the final animated result.

20:24.140 --> 20:29.580
So you've got an obvious problem there of synchronizing, especially if the OEs aren't

20:29.580 --> 20:33.380
sitting right next to the animators, which was our situation.

20:33.380 --> 20:44.100
So at least early on, you constantly had object engineers going back to talk to the animators,

20:44.100 --> 20:47.660
trying to get them to run through, you know, show them in Maya where certain events were,

20:47.660 --> 20:50.420
trying to figure out what they were missing.

20:50.420 --> 20:55.180
It was a big process problem.

20:55.180 --> 20:59.860
As a result of this, one of the OEs actually, this Skunkworks project, produced this little

20:59.860 --> 21:06.140
tool called Clockwork, which basically sucked in all the animations, read all the tags and

21:06.140 --> 21:10.460
provided a little browser, which made it very easy to figure out what the animations were

21:10.460 --> 21:16.580
doing and where all the timing events were.

21:16.580 --> 21:22.540
So Edith is the visual scripting language that drives all of this.

21:22.540 --> 21:26.980
And actually, one of the main learning experiences from The Sims 2 and probably the previous

21:26.980 --> 21:31.420
expansion packs too, is that for this situation anyway, visual scripting just does not work

21:31.420 --> 21:32.900
very well.

21:32.900 --> 21:37.220
You give up an awful lot for a nice visual scripting language.

21:37.220 --> 21:42.820
There's no revision history in Perforce, no good search and replace, all kinds of things

21:42.820 --> 21:47.900
you just take for granted with script text scripting languages you lose.

21:47.900 --> 21:49.940
And that becomes pretty frustrating.

21:49.940 --> 21:54.700
So I did a little survey at the end of the project and pretty much, well, to a man, all

21:54.700 --> 22:00.740
OEs wanted to move away from it.

22:00.740 --> 22:05.260
The one good thing about a visual scripting language is that it pretty much enforces having

22:05.260 --> 22:12.820
a good debugger, although I gather it was broken for a good part of the time.

22:12.820 --> 22:17.300
So we even tried to use Edith a bit on Sim City 4 of all things, which didn't work out

22:17.300 --> 22:19.020
at all well.

22:19.020 --> 22:24.900
So our studio can say this now, except we're basically going to ditch Edith and attempt

22:24.900 --> 22:32.020
to use Lua in the same capacity.

22:32.020 --> 22:33.060
Engineering.

22:33.060 --> 22:48.980
We had about 28 engineers at peak, starting off with about 5 or 10 for the first few years.

22:49.540 --> 22:53.180
So as I said before, 1.1 million lines of code.

22:53.180 --> 22:55.660
We have quite a lot of shared code at Maxis.

22:55.660 --> 23:02.580
There's this framework code that has been shared, I believe started with SimCopter,

23:02.580 --> 23:09.580
and has been used in all sorts of titles since then, SimCity 3000, SimCity 4, Sims Online,

23:09.580 --> 23:15.260
the original Sims, all the UI code came from this framework.

23:15.260 --> 23:21.100
And we also had a lot of engineering-generated scripts, mostly materials, but also things

23:21.100 --> 23:26.940
like cameras and catalog and light tuning.

23:26.940 --> 23:31.540
We had one dedicated engineer to basically what we call the WorldDB, which is the world

23:31.540 --> 23:35.900
representation, so the train and so on.

23:35.900 --> 23:40.340
And that was our main link between gameplay and the engine.

23:40.340 --> 23:45.540
There was a lot of talk early on in Sims 2 about moving away from the tile-based system

23:45.540 --> 23:49.660
that it subsists on, and quite frankly, we go through this with every new title, and

23:49.660 --> 23:50.660
it's the same with SimCity.

23:50.660 --> 23:54.980
All of our games are kind of tile-based, aesthetically you'd like to move away from that.

23:54.980 --> 23:59.180
The reason that we stay with tile-based systems is just UI and kind of gameplay.

23:59.180 --> 24:04.020
We would have to drastically rewrite the simulator to be able to move away from that.

24:04.020 --> 24:07.900
This actually gave us a problem later on, because we had this nice generalized WorldDB

24:07.900 --> 24:20.900
code, and it was quite hard to make it efficient in terms of a tile-based structure.

24:20.900 --> 24:24.740
So routing was one of the biggest complaints about Sims 1.

24:24.740 --> 24:28.900
You had this party syndrome where you'd have 20 people in a lot, all trying to get somewhere

24:28.900 --> 24:31.340
at some time, and basically stacking up.

24:31.980 --> 24:38.660
Early on, we contracted a company to write a replacement for the Sims 1 router, but the

24:38.660 --> 24:41.180
results weren't that fantastic.

24:41.180 --> 24:44.580
And even if they had been, there was still a problem in that the router is really essential

24:44.580 --> 24:46.860
to gameplay in The Sims.

24:46.860 --> 24:52.380
So it's really something that we needed to own and kind of iterate on with the simulation

24:52.380 --> 24:53.380
engineers.

24:53.380 --> 24:59.860
So instead, we dedicated an engineer to that, and I think that was a very good decision.

25:00.860 --> 25:05.500
That's just a quick list of some of the features in it.

25:05.500 --> 25:10.540
Probably what makes it most different from other routers is just some of these, the last

25:10.540 --> 25:15.940
two extra features, and also the fact that the routing paths tended to vary a lot based

25:15.940 --> 25:19.260
on what the simulator was telling the routing system.

25:19.260 --> 25:23.900
So there was very kind of tight coupling there.

25:23.900 --> 25:30.900
Something pretty standard, blended animations, two bone IK, and a bunch of extra features

25:30.900 --> 25:32.940
to handle certain things.

25:32.940 --> 25:34.780
Standard reach is probably the biggest one.

25:34.780 --> 25:39.460
We obviously have the problem of Sims walking around and having to pick up cups and put

25:39.460 --> 25:44.980
stuff in the microwave and all the rest of it.

25:44.980 --> 25:49.380
We also had an effect system, which started off as just a particle system on SimCity 4,

25:49.860 --> 25:55.700
and it's now kind of grown into this monster prototyping system whereby, so it's all script-based,

25:55.700 --> 26:02.060
hot-loaded, and kind of hierarchically composed.

26:02.060 --> 26:06.660
So a lot of the stuff in Sims, the dynamic geometry that you see is almost all the effect

26:06.660 --> 26:10.900
system, and that kind of includes things like the thought balloons, a lot of the UI that

26:10.900 --> 26:14.380
isn't text-based.

26:14.380 --> 26:17.620
So I just have a little example here of, say, the fish.

26:17.620 --> 26:22.220
So if you play the Sims, you can buy this little fish tank, and the little fish in the

26:22.220 --> 26:27.260
aquarium are all run by this effect system, basically a combination of random walks and

26:27.260 --> 26:39.100
colliders and quite elaborate state transitions.

26:39.100 --> 26:41.900
We tried to move forward the neighborhood quite a bit on Sims 2.

26:41.900 --> 26:46.420
In Sims 1, the neighborhood was screen capture, essentially, screen render.

26:46.420 --> 26:51.740
We tried to make it much more of like a sandbox in Sims 2, much like the lot.

26:51.740 --> 26:55.660
So the hardest technical problem there was just kind of imposterizing lots dynamically

26:55.660 --> 26:57.340
on the fly.

26:57.340 --> 27:01.900
When you exit from a lot, then we build a little imposter that kind of captures roughly

27:01.900 --> 27:09.420
what it looks like, and we also import all of our terrains from SimCity, which made production

27:09.420 --> 27:12.260
actually of, you know, a variety of different neighborhoods pretty easy.

27:12.260 --> 27:17.260
There's nothing like having one of your other games provide quite an elaborate production

27:17.260 --> 27:23.740
tool to speed things up.

27:23.740 --> 27:28.100
I talked before about the lighting system and how it had to handle, basically, a dynamic

27:28.100 --> 27:29.100
environment.

27:29.100 --> 27:32.100
How we did this is that lighting was room-based.

27:32.100 --> 27:40.900
We had code that would break up our buildings into rooms and portals between each room,

27:40.900 --> 27:44.020
and then we basically just did the obvious things.

27:44.020 --> 27:50.460
The most elaborate part of the system is just that portals can transmit light.

27:50.460 --> 27:54.580
So you can see in the left-hand picture here that actually the only light that's being

27:54.580 --> 28:01.100
specified here is that the artist has, lighting tuner has specified the outdoor lighting conditions,

28:01.100 --> 28:04.780
and then the light makes its way in through the portals all the way into that inner room

28:04.780 --> 28:16.420
and a set of fixed function lights is constructed there to kind of light those chairs correctly.

28:16.420 --> 28:20.380
So yeah.

28:20.380 --> 28:29.420
Early on we decided we absolutely needed over-bright lighting to get, you know, a decent look.

28:29.420 --> 28:35.140
That actually bit us later in a non-obvious way.

28:35.140 --> 28:41.660
So basically we had a lot of problems when tuning the lighting system, avoiding blow-out,

28:41.660 --> 28:45.700
because of the way that windows could be placed almost anywhere and object lights could be

28:45.700 --> 28:51.340
placed almost anywhere, preventing a sim from basically blowing out completely to white

28:51.340 --> 29:01.100
or a nearby bed was problematic.

29:01.100 --> 29:05.060
Shadows are, the only really tricky thing about the shadows was just the sheer number

29:05.060 --> 29:07.020
of them.

29:07.020 --> 29:10.820
So basically every object has to have a shadow.

29:10.820 --> 29:15.420
The terrain and house were actually not done using traditional kind of graphics, projective

29:15.420 --> 29:16.420
texturing shadows.

29:16.740 --> 29:21.580
There was a fast CPU side algorithm that gave us back a little map that we could look up

29:21.580 --> 29:26.220
for any location in the world, you know, is that in shadow depending on the terrain or

29:26.220 --> 29:32.740
the house, which suited our target hardware.

29:32.740 --> 29:36.540
So our target hardware was very much kind of DX7 level, so we had to keep most of our

29:36.540 --> 29:40.780
graphics tricks pretty simple.

29:40.780 --> 29:49.100
The other thing we used was these GUOBs, which stand for Generic Under Object Blobs.

29:49.100 --> 29:55.500
These are basically just cards like you saw in the ATI demo that we pre-render to look

29:55.500 --> 30:00.220
originally like blobs, but after that quite nice, you know, we did Maya software rendering

30:00.220 --> 30:04.500
to get a nice kind of ambient type shadow indoors.

30:04.500 --> 30:06.660
This is most important for things like contact shadows.

30:06.660 --> 30:10.660
If you have a painting against a wall, it really looks much better if you kind of include

30:10.660 --> 30:17.580
the contact shadow that it's casting against that wall, it helps kind of seat it.

30:17.580 --> 30:25.100
So the graphics engine was a curiosity in some ways.

30:25.100 --> 30:29.540
It was written as a completely generic engine that, you know, tried to disregard anything

30:29.540 --> 30:34.500
to do with gameplay in the Sims engine itself, and it was written as a completely generic

30:34.500 --> 30:36.300
scene graph.

30:36.300 --> 30:40.860
So the way you assemble your scene is basically by creating normal, you know, Maya style.

30:40.860 --> 30:45.620
Your Sim might be a hundred different transform nodes that you've read off disk and you insert

30:45.620 --> 30:47.780
that into the scene.

30:47.780 --> 30:53.660
If you want to find a node on that Sim, you do a traversal over that sub-branch and look

30:53.660 --> 30:55.960
for tags and so on.

30:55.960 --> 31:03.420
So basically a lot of caching went on to make this all work properly.

31:03.420 --> 31:09.900
And it just seems like a fundamentally bad idea.

31:09.900 --> 31:13.940
This kind of thing really belongs in your art pipeline, I would claim.

31:13.940 --> 31:18.860
Because just the overhead and the extra complexity in dealing with a scene graph, you know, actually

31:18.860 --> 31:24.700
in gameplay code was enough that I think it brought down the productivity of the actual

31:24.700 --> 31:32.020
game programmers quite a bit.

31:32.020 --> 31:35.620
So we had graphics performance problems with this, as you might imagine, and plus just

31:35.620 --> 31:36.620
our batch count.

31:36.620 --> 31:41.140
So we're rendering a lot with a whole bunch of, you know, hundreds, two hundreds of different

31:41.140 --> 31:47.140
objects in it, and each object might have different subsets, and plus you have the walls

31:47.140 --> 31:50.420
and the roof and all the rest of it.

31:50.420 --> 31:59.060
So this was a problem for us early on, and eventually the solution we turned to was a

31:59.060 --> 32:01.700
very old and hoary one.

32:01.700 --> 32:05.540
So SimCity uses Dirty Rex.

32:05.540 --> 32:10.420
Basically dynamic objects are re-rendered per frame, and anything that we know is static

32:10.420 --> 32:15.100
we just hold over from the previous frame.

32:15.100 --> 32:21.340
So I thought SimCity would be the last game that ever shipped using that technology, but

32:21.340 --> 32:22.380
that's the way it went.

32:22.380 --> 32:30.220
We had an initial kind of prototype system just to get working on low-end hardware, Dirty

32:30.220 --> 32:36.780
Rex system, and this eventually morphed into over a number of months a full SimCity 4 style

32:36.780 --> 32:46.860
dual-layer system, and this caused us no end of problems just because it was retrofitted.

32:46.860 --> 32:50.900
There were all sorts of subsystems that basically needed a lot of extra code added so that we

32:50.900 --> 32:59.860
could identify when things had changed and where they had changed.

33:00.500 --> 33:06.460
Some of this came about just because our target platforms were so broad and so low on the

33:06.460 --> 33:07.620
minimum ends.

33:07.620 --> 33:17.420
So we tried to support actual non-TNL commodity Intel hardware, you know, the 815, 845, and

33:17.420 --> 33:22.220
basically the broad hump of the cards we were trying to support or machines we were trying

33:22.220 --> 33:26.300
to support were machines with DX7 level cards.

33:26.300 --> 33:30.620
There's a whole bunch of extra slides and commentary in the proceedings that I encourage

33:30.620 --> 33:36.100
you to read if you're interested in this, but given that a lot of this is kind of not

33:36.100 --> 33:42.860
of interest going forward, I figured I'd better skip it.

33:42.860 --> 33:48.700
Apart from this slide, so game configuration, this is a real headache, targeting that many

33:48.700 --> 33:53.220
target platforms.

33:53.220 --> 33:58.540
We had a reasonably elaborate system on SimCity to handle all the different possible configurations

33:58.540 --> 34:03.180
and driver bugs and all the rest of it, and we basically amped that up by about a factor

34:03.180 --> 34:06.060
of two, I think.

34:06.060 --> 34:09.940
So our experience has been, I'm sure it's the same elsewhere, is that relying on cap

34:09.940 --> 34:15.100
bits or any kind of, relying on the card to tell you what it can do or not is just, does

34:15.100 --> 34:17.380
not work.

34:17.380 --> 34:21.500
So basically we build a lot of logic into scripts that tell us, you know, what cards

34:21.500 --> 34:30.300
can support which and where to set various level of detail parameters.

34:30.300 --> 34:36.740
So memory handling, this is a PC title, so as usual we're pretty fast and loose about

34:36.740 --> 34:38.020
memory.

34:38.020 --> 34:45.300
A lot of STL, very similar to Charles' description of what they were using at Oddworld, almost

34:45.300 --> 34:52.260
all vectors, not always efficient, but we can usually pick up cases where it isn't efficient

34:52.260 --> 34:58.140
with profilers, and our main obsession with a lot of this stuff is just leak prevention.

34:58.140 --> 35:02.100
Because we have to scale the game over across, across a number of platforms, it's much more

35:02.100 --> 35:08.100
difficult to kind of hit, you know, we can't just hit a particular memory target, we probably

35:08.100 --> 35:12.660
have hit four or five, depending on the different platforms that we're targeting, you know,

35:12.660 --> 35:18.740
how much memory the system has, whether it's running a card that's going to require backing

35:18.740 --> 35:22.300
from system memory, or the rest of it.

35:22.300 --> 35:28.740
We also use ref counting a lot, and interfaces, and similar kind of smart pointers, auto ref

35:28.740 --> 35:33.180
count is basically the same kind of smart pointer thing that Charles was talking about.

35:33.180 --> 35:39.380
And for anything that really needs efficient allocation, we have custom allocators to do,

35:39.380 --> 35:48.420
you know, similar pools, and things like that.

35:48.420 --> 35:53.860
We have some interesting observations over the last couple of games that we've shipped.

35:53.860 --> 35:57.860
Basically it's reasonably easy to track down leaks these days.

35:57.860 --> 36:01.900
We've done a number of code sweeps, and kind of added up the percentages of where we find

36:01.900 --> 36:07.260
the problems, and if you want to find a memory leak, just search for a new or delete in a

36:07.780 --> 36:11.860
So as a result, we have almost no, we avoid this.

36:11.860 --> 36:15.060
We used to have a lot of manual ref counting, we're replacing all of that, because that's

36:15.060 --> 36:16.060
another hot spot.

36:16.060 --> 36:20.660
Not quite as bad, but the trouble with manual ref counting is that it tends to be very fragile,

36:20.660 --> 36:26.620
so we'll come along later and add some code, and break an existing working system.

36:26.620 --> 36:31.620
And finally, we avoid the kind of ref count loops by having very kind of solid init and

36:31.620 --> 36:35.580
shutdown approaches.

36:35.580 --> 36:40.460
So basically be sure to init and shutdown all classes, and it's absolutely a requirement

36:40.460 --> 36:45.260
that a shutdown releases all member auto-ref pointers.

36:45.260 --> 36:47.740
I thought this would be interesting.

36:47.740 --> 36:54.580
In terms of the biggest leak that we had during finaling, SimCity 4 was actually the Lua garbage

36:54.580 --> 36:57.940
collection, which I find kind of ironic.

36:57.940 --> 37:03.220
There was a race condition in the Lua code that led to massive memory leaks in a particular

37:03.220 --> 37:04.220
situation.

37:04.220 --> 37:09.500
In Sims 2, we discovered in the last couple of weeks that someone had left on a particular

37:09.500 --> 37:18.660
logging system, and that basically that log data was all being accumulated in a big buffer.

37:18.660 --> 37:22.940
Another of the interesting observations from Sims 2 is that we, although we got well back

37:22.940 --> 37:28.260
from this, you know, by the time we shipped, we ran into situations as we entered finaling

37:28.260 --> 37:32.620
where we were actually running out of virtual memory space.

37:32.620 --> 37:38.980
So PC games have long used virtual memory as kind of like a crutch to avoid having to

37:38.980 --> 37:43.820
be too careful about the amount of memory that you use, but that kind of free lunch

37:43.820 --> 37:47.420
is running out.

37:47.420 --> 37:54.300
Basically that address space is getting smaller and smaller depending on DLs and things like

37:54.300 --> 37:57.300
that.

37:57.300 --> 37:59.660
So I'll just give you an example.

37:59.660 --> 38:03.020
This is a little snapshot of the Sims 2.

38:03.020 --> 38:07.100
This is actually after we solved our big leaks that were causing us to run out of virtual

38:07.100 --> 38:12.140
memory, but you can still see that big blue band in the middle is all of the allocation

38:12.140 --> 38:19.900
from just our, basically the engine and the scene graph and the first layer of game code

38:19.900 --> 38:27.500
over top of that, and we have a lot of allocation churn from that, and as a result, a lot of

38:27.500 --> 38:37.740
kind of space that we've touched and then released again.

38:37.740 --> 38:38.740
So resource management.

38:38.740 --> 38:42.500
I'm not going to talk about this too much because I think it's been covered a fair bit

38:42.500 --> 38:45.060
in previous talks.

38:45.060 --> 38:49.820
We have a reasonably kind of conservative key-based system that we use at Maxis.

38:49.820 --> 39:00.860
The only real wrinkle that came here is that the initial Sims 2 team, excuse me, weren't

39:00.860 --> 39:07.460
really familiar with it, and first of all kind of overused resources, and second of

39:07.460 --> 39:15.100
all tried to do an alternate kind of name-based system and wound up kind of hashing strings

39:15.100 --> 39:20.380
into 32-bit UIDs.

39:20.380 --> 39:24.020
Now you can kind of get away with that, and we do get away with it on, say, console titles

39:24.020 --> 39:28.380
at Maxis where you have a smallish number of resources, 5,000, 10,000 or something,

39:28.380 --> 39:36.100
but with our resource count, you're just bound to get collisions, and indeed we did,

39:36.100 --> 39:40.540
and we also had the usual kind of custom content headaches of people, you know, someone in

39:40.540 --> 39:47.420
India creates a skin, someone in England creates a skin, and then they upload their skins and

39:47.420 --> 39:56.780
someone in America downloads them both, and you have to be careful about resource collisions.

39:56.780 --> 40:01.060
So we basically brute-forced both of these problems.

40:01.060 --> 40:08.180
The extending the instance ID to 64 bits, basically that solution came about just because

40:08.180 --> 40:13.380
we'd painted ourselves into a corner, you know, with any amount of kind of simple straightforward

40:13.380 --> 40:16.860
planning at the beginning of the project, we could have avoided the situation, but we

40:16.860 --> 40:23.260
wound up very late having to deal with these collisions, and in the end making that kind

40:23.260 --> 40:32.620
of brute-force change was the easiest thing to do.

40:32.620 --> 40:33.620
Configuration management.

40:33.620 --> 40:38.140
We're starting to get quite a big configuration management team.

40:38.140 --> 40:42.420
It seemed quite similar to what Chris was talking about earlier with Bungie.

40:42.420 --> 40:45.380
We had like six to eight engineers on our configuration management team, kind of shared

40:45.380 --> 40:49.100
between Sims 2 and a few other projects.

40:49.100 --> 40:53.380
We have a system at Maxis where we test the game by kind of running commands through a

40:53.380 --> 40:56.180
command console, which has proven very effective.

40:56.180 --> 41:01.580
So basically we can write a whole bunch of unit tests that run the game through its paces

41:01.580 --> 41:06.860
and then check for certain conditions, and, you know, once we build up a library of tests

41:06.860 --> 41:13.900
like that, we can be running them constantly, week in and week out.

41:13.900 --> 41:18.940
Before each check-in, even early on in development, you're required to run a simple sniff test

41:18.940 --> 41:25.380
to basically make sure that the game was reasonably functional.

41:25.380 --> 41:27.460
We used DevTrack.

41:27.460 --> 41:32.020
We had our own custom tool for rolling out builds.

41:32.020 --> 41:38.140
So basically it turns out to be a bad idea to roll out development builds where you have

41:38.140 --> 41:41.460
hundreds of thousands of files incrementally.

41:41.460 --> 41:42.860
It's a bad idea for the network.

41:42.860 --> 41:47.540
It's a bad idea for File.io.

41:47.540 --> 41:54.820
We probably would have been better off just releasing zip files of daily builds.

41:54.820 --> 41:59.860
We had a similar facility discussed before here where if the game crashed, it would send

41:59.860 --> 42:07.740
off a stack dump and all sorts of other annotations about the simulator and the animation system,

42:07.740 --> 42:10.380
and then these would be gathered and kind of published on a website.

42:10.380 --> 42:14.340
So you could kind of go to that website on a particular day and see, you know, the most

42:14.340 --> 42:23.060
common crashes and follow the links and actually get the place in the code where it crashed.

42:23.060 --> 42:26.180
We had quite an elaborate source control set up.

42:26.180 --> 42:27.940
So we actually had two branches.

42:27.940 --> 42:32.860
The engineers worked on this DevLan branch, and that actually went through a testing cycle

42:32.860 --> 42:36.340
before it was then integrated across the main line.

42:36.340 --> 42:40.460
The main line was also then tested before that was in turn released through this Robby

42:40.460 --> 42:44.140
tool to production and art.

42:44.140 --> 42:48.380
So it turns out this is just way too much process for a game except when you're on the

42:48.380 --> 42:54.580
last few months, and one of the common bits of feedback after the game shipped was basically

42:54.580 --> 42:59.140
there was way too much turnaround between, say, an engineer checking in a feature and

42:59.140 --> 43:06.700
an artist or a scripter seeing the result.

43:06.700 --> 43:12.900
It could be up to a week, so pretty bad.

43:12.900 --> 43:21.820
The other problem was just that I forgot to mention on the other slide is that our publishing

43:21.820 --> 43:24.060
tool had no rollback facility.

43:24.060 --> 43:29.020
So if we did, if in spite of all this process, we did manage to publish a bad build that

43:29.020 --> 43:34.500
was broken in some way, that would basically stop people for a day or so.

43:34.500 --> 43:38.860
So lessons learned.

43:38.860 --> 43:43.820
So everything was kind of nice and peaceful after shipping, but at some point you have

43:43.820 --> 43:49.540
to, like, sit down and figure out what you can change.

43:49.540 --> 43:54.620
So the thing that went right was basically art and content like that.

43:54.620 --> 43:58.640
We had all sorts of the usual process problems, but when it came down to it, we could always

43:58.640 --> 44:00.860
hit art block.

44:00.860 --> 44:04.180
And one of the things here is just that we have a lot of crossover with the animation

44:04.180 --> 44:05.180
industry these days.

44:05.580 --> 44:09.100
Like I said, our technical art director was from Pixar and was used to setting up a lot

44:09.100 --> 44:13.500
of the pipeline systems that we use.

44:13.500 --> 44:16.340
And we also are hiring from the animation industry now.

44:16.340 --> 44:23.060
And they've been dealing with these problems for quite a while.

44:23.060 --> 44:27.820
The things that went wrong in art are basically going back to design again.

44:27.820 --> 44:30.980
There were a lot of late changes to design.

44:30.980 --> 44:36.940
We had, I think at one stage, two or three art directors, and also executives and even

44:36.940 --> 44:40.380
producers would throw in commentary about how something should look.

44:40.380 --> 44:43.860
So there was kind of a lot of trashing going on there.

44:43.860 --> 44:50.140
There were some communication issues, the usual thing with a team that big.

44:50.140 --> 44:54.780
The tighter code turnaround that I was talking about, and just generally there wasn't a lot

44:54.780 --> 45:01.780
of communication between the engineering team and art, apart from a few pathways.

45:01.780 --> 45:06.660
The kind of person we're having trouble hiring is kind of technical art types.

45:06.660 --> 45:11.780
So we're starting to see this need for the kind of technical director role from animation

45:11.780 --> 45:16.580
again, someone who's kind of an artist, but is also very good at scripting or whipping

45:16.580 --> 45:22.060
up little useful pipeline tools.

45:22.060 --> 45:24.740
This was probably our biggest lesson.

45:24.740 --> 45:31.100
It's just, the later you make a design change, especially if it's a reasonably big one, the

45:31.100 --> 45:37.180
bigger the cost to engineering, art, production, just everyone.

45:37.180 --> 45:43.380
So we were kind of between a rock and a hard place because we had to get that design right.

45:43.380 --> 45:44.380
And it wasn't right.

45:44.380 --> 45:49.180
You know, Maxis has been through this before with the Simsville title that never shipped.

45:49.180 --> 45:53.180
The reason it never shipped is basically it looked gorgeous, but there was no gameplay

45:53.180 --> 45:56.060
there.

45:56.060 --> 46:00.940
So one of our biggest issues is basically how do you address that kind of thing, which

46:00.940 --> 46:04.380
I'll get to a bit later.

46:04.380 --> 46:06.180
Engineering lessons, we had a few.

46:06.180 --> 46:10.660
The engineering process was pretty chaotic.

46:10.660 --> 46:16.380
So we had a bit of a problem with, you know, a large section of engineering mostly concentrating

46:16.380 --> 46:18.940
on the graphics engine and not on the game.

46:18.940 --> 46:22.180
In fact, the game got neglected to some extent.

46:22.180 --> 46:27.060
We had an ongoing problem, especially early on in development, of not having enough people

46:27.060 --> 46:31.860
actually working on the simulator and the various things that were going to be delivering

46:31.860 --> 46:34.060
all this new gameplay.

46:34.060 --> 46:38.100
It's kind of indicative that the research team that went for two or three years, only

46:38.100 --> 46:42.660
about half the time they ever had an engineer actually working on the Sims simulator code

46:42.660 --> 46:44.420
base.

46:44.460 --> 46:47.900
No one else was working on the graphics engine.

46:47.900 --> 46:51.620
Like I mentioned before, a scene graph really belongs in the pipeline.

46:51.620 --> 46:57.220
I can go into that in more detail afterwards, but this basically did not work for us particularly

46:57.220 --> 47:05.220
well, just because it kind of sucks, well, there's the complexity issue, and then you

47:05.220 --> 47:12.940
get into problems with fragility, where you're persisting, say, a scene graph branch that's

47:12.940 --> 47:16.740
supposed to be matching, say, your code, your gameplay state, and if those get out

47:16.740 --> 47:21.940
of sync, then you can run into all sorts of problems.

47:21.940 --> 47:27.220
The resource problem and a few other simple things like that kind of taught us don't ignore,

47:27.220 --> 47:29.540
you know, your existing shipping hardened code.

47:29.540 --> 47:34.140
Don't think it's a good idea to spend engineering time trying to come up with a better system

47:34.140 --> 47:39.320
where you already have a decent one.

47:39.320 --> 47:42.300
Don't contract out core gameplay components.

47:42.300 --> 47:45.340
That was the lesson from the routing.

47:45.340 --> 47:52.500
And from engineering, basically more urgency, part of the result of having some engineers

47:52.500 --> 47:59.180
kind of detached from the actual game is that there wasn't a lot of urgency, and that was

47:59.180 --> 48:05.060
actually where most of our senior graphics, sorry, our senior engineers kind of fell,

48:05.060 --> 48:08.860
so that meant there was kind of a lack of urgency from the top of the team to try and

48:08.860 --> 48:15.540
get this thing out the door.

48:15.540 --> 48:17.240
This is a transporter accident.

48:17.240 --> 48:24.660
We have hundreds of these, usual kind of skinning accident.

48:24.660 --> 48:29.740
So I'm going to run through a couple of slides of drilling down onto what things went wrong

48:29.740 --> 48:33.100
in engineering, just because it's my area.

48:33.100 --> 48:38.420
We had a lot of over-engineering, and I list a completely insane example here, but there

48:38.420 --> 48:40.620
are other examples almost this insane.

48:40.620 --> 48:50.940
So the concept of a pixel format, you know, ARGB, 5651, sorry, 5551, all the rest of it.

48:50.940 --> 48:52.700
Something you'd normally represent by an enum.

48:52.700 --> 49:00.060
Well, we had five interfaces, I think, and 2,200 lines of code, just to represent that

49:00.060 --> 49:01.060
concept.

49:01.060 --> 49:07.740
We had some people who are very enamored of C++ and of COM, and this kind of led to what

49:07.740 --> 49:11.820
I call bubble wrap syndrome, which is where you have a reasonably simple kind of core

49:11.820 --> 49:16.780
game concept or engine concept that you want to be able to manipulate, but has so much

49:16.780 --> 49:22.380
layering over the top of it that it feels like you just can't push through and directly

49:22.380 --> 49:25.180
do that thing.

49:25.180 --> 49:31.540
We're gradually transitioning to renderware at Maxis, and so far I'm liking that a lot

49:31.540 --> 49:37.380
better because it has this kind of toolkit concept, where basically you grab bits of

49:37.380 --> 49:44.860
the code and use that directly.

49:44.860 --> 49:51.140
So Charles said not to diss C++, but there are some areas of C++ that you just have to.

49:51.140 --> 49:55.580
So our math library was written using template metaprogramming, which was all the rage a

49:55.580 --> 49:58.220
few years back.

49:58.660 --> 50:03.540
This was kind of a common problem on the Sims 2 engineering-wise, which is if people implement

50:03.540 --> 50:07.660
new approaches, then they should always profile them and show that they're actually better.

50:07.660 --> 50:12.220
So unfortunately no one ever did this, and it turns out that even with this template

50:12.220 --> 50:16.980
metaprogramming idea, which is that you actually wind up with more efficient code because you're

50:16.980 --> 50:23.900
throwing all of the math instructions at the compiler to do with what it will, we got a

50:23.900 --> 50:29.860
decrease in performance over just a normal standard math library because we're basically

50:29.860 --> 50:34.980
blowing the compiler's mind.

50:34.980 --> 50:39.740
Any compiler is going to give up after a certain amount of inline depth.

50:39.740 --> 50:41.500
That actually wasn't the problem.

50:41.500 --> 50:46.020
The problem with this approach, which kind of leads to one of these non-obvious things

50:46.020 --> 50:49.840
you have to be careful about, is its impact on debug speed.

50:49.840 --> 50:59.560
So this approach to math vector library multiplied by a lot the number of function calls in any

50:59.560 --> 51:01.040
particular math operation.

51:01.040 --> 51:07.560
So if I remember right, a vec4 addition wound up being 20 function calls in debug.

51:07.560 --> 51:08.840
That's with asserts off.

51:08.840 --> 51:12.040
With asserts on, it was more like 40 or 50.

51:12.040 --> 51:19.440
So when we did kind of a reverse patch job and tried to mitigate some of this stuff,

51:19.440 --> 51:24.880
we found that the overall impact on debug speed was something like 75% on our core engine

51:24.880 --> 51:29.000
routines.

51:29.000 --> 51:34.000
So what this is pointing to is just you really have to be careful in the early stages of

51:34.000 --> 51:43.080
your project in terms of having people who are experienced and practical because the

51:43.080 --> 51:50.720
stuff they do will impact the later programmers immensely.

51:50.720 --> 51:53.000
Another problem was just API churn.

51:53.000 --> 51:59.760
So because we wound up with these kind of ghettoized, well not ghettoized, but separated

51:59.760 --> 52:05.560
engineering teams, one of which was highly dependent on the other, but weren't really

52:05.560 --> 52:11.040
talking to each other, you have this problem of API churn, where there was a number of

52:11.040 --> 52:15.320
low-level APIs that were being changed constantly to meet some measure of conceptual purity

52:15.320 --> 52:18.320
on the engine side.

52:18.320 --> 52:31.040
That led to, you know, the obvious problems you might imagine.

52:31.040 --> 52:36.840
So adding this all up, my point of view basically came down to a lack of productivity.

52:36.840 --> 52:44.240
At some point you have to ship the game and you have to be careful that you don't attempt

52:44.240 --> 52:49.840
too much kind of research-y code generation before doing that.

52:49.840 --> 52:56.760
And one of our learning experiences is that we, by kind of taking the conceptually pure

52:56.760 --> 53:00.840
approach and trying to do things right, we actually lost a lot of opportunities.

53:00.840 --> 53:06.820
So the problem with taking this kind of approach is that you run out of time to basically finish

53:06.820 --> 53:09.820
off stuff, or even get anything working.

53:09.820 --> 53:17.500
So for instance, we have a lot of well-authored normal maps for our sims, but due to the slowness

53:17.500 --> 53:23.580
of the whole normal map process, in the last couple of weeks of the project, the decision

53:23.580 --> 53:28.580
was made to basically make them only ever show up on a 128-meg cart, because no one

53:28.580 --> 53:33.100
ever actually implemented, you know, any kind of normal map compression, or even enough

53:33.100 --> 53:37.980
code to be able to drop the level of detail of the normal map.

53:37.980 --> 53:41.140
I've been through the static LODs.

53:41.140 --> 53:46.700
We dropped our main shader path also in the last couple of weeks, for similar reasons.

53:46.700 --> 53:53.620
It's just, you have a tight deadline.

53:53.620 --> 53:56.700
You've got to keep close to the middle when you're making a lot of these decisions, or

53:56.700 --> 54:04.460
you just will run out of time.

54:04.460 --> 54:09.420
So some of that stuff is what I'm referring to when I say we shipped, because it kind

54:09.420 --> 54:12.460
of amazes me that we did sometimes.

54:12.460 --> 54:16.420
So that's lessons learned, you know, how we actually, what are we going to change in what

54:16.420 --> 54:17.420
we're doing.

54:17.420 --> 54:22.060
Like I said, one of our biggest problems was design churn, so what we're doing is learning

54:22.060 --> 54:24.840
about pre-production.

54:24.840 --> 54:28.840
This is something that other areas of EA have been reasonably big on, but we never have

54:28.840 --> 54:32.640
it in Maxis, just because we haven't been in that arena until, say, Sims 2, or maybe

54:32.640 --> 54:37.520
The Sims Online, where you have massive teams, and basically getting everything through the

54:37.520 --> 54:43.560
sausage factory and out the other side is, you really have to do it in order.

54:43.560 --> 54:47.520
So the whole purpose of pre-production is that you do all of your research then, all

54:47.520 --> 54:53.000
of your prototyping, make as many design decisions as possible that are going to stick, solely

54:53.160 --> 54:59.720
because every one of those that you do, you're reducing the risk later on.

54:59.720 --> 55:03.040
And then, you know, theoretically you go through the slow ramp up to full production, slowly

55:03.040 --> 55:09.520
throw more engineers at it, and it all works marvelously.

55:09.520 --> 55:10.520
So we'll see how that goes.

55:10.520 --> 55:11.520
For a project your size, how many people do you think would be on pre-production?

55:11.520 --> 55:12.520
10 to 20.

55:12.520 --> 55:13.520
So why didn't we do this before?

55:13.520 --> 55:29.840
Because, after all, the early Sims 2 research team was about 5'10", 20'10", it kind of bounced

55:29.840 --> 55:34.360
around depending on what other project was going on at the time.

55:34.360 --> 55:36.460
Basically we just didn't have the right mindset.

55:36.460 --> 55:42.360
We treated it as a research team, and most of that research went into graphics research,

55:42.360 --> 55:46.680
or very limited new idea research that wasn't actually implemented and tested.

55:46.680 --> 55:52.040
So the key is really, don't just make design decisions, get in there and prototype them,

55:52.040 --> 55:55.240
and do some gameplay prototypes and actually play through and see whether it's going to

55:55.240 --> 56:01.640
work.

56:01.640 --> 56:08.080
In terms of some of our communication issues, you know, these issues are not due to anything

56:08.080 --> 56:13.680
particular at Maxis, it's just the problem of having teams that large and getting communication

56:13.680 --> 56:15.600
from one place to another.

56:15.600 --> 56:19.040
We really kind of have to change the way that we work.

56:19.040 --> 56:22.440
So we're trying this kind of pod style of working where we move much more to a kind

56:22.440 --> 56:28.000
of a SWAT team approach that we have kind of used in the past.

56:28.000 --> 56:32.680
So the goal is to get into these kind of small groups where the iteration turnaround time

56:32.680 --> 56:45.120
is really fast and there's not a lot of kind of cross process overhead to slow people down.

56:45.120 --> 56:49.040
This is a small example of that kind of thing that happened on The Sims 2.

56:49.040 --> 56:56.080
So for the original ship deadline of January 2004, we actually cut swimming pools, because

56:56.080 --> 57:01.720
we had none of it done, and it was going to be quite a bit of work, which was almost unbelievable.

57:01.760 --> 57:05.560
Swimming pools are an essential part of The Sims game, because they're how you kill Sims.

57:05.560 --> 57:11.200
I mean, you throw them in the pool, you take away the ladder, and they die.

57:11.200 --> 57:19.000
So we got a skunkworks team together to help save them, driven by a producer, all completely

57:19.000 --> 57:22.240
outside the normal process rules.

57:22.240 --> 57:25.040
Simplest things we could do, basically just get something in there and get it working,

57:25.040 --> 57:29.200
you know, a small number of people, and I think we did a really good job on kind of

57:29.200 --> 57:30.800
getting that all in there and getting it working.

57:30.880 --> 57:36.080
It was kind of a lesson in terms of how much kind of stepping outside the big meat grinder

57:36.080 --> 57:41.920
process and just having a small SWAT team working on a particular feature could work.

57:41.920 --> 57:49.800
And of course we added more stuff to that once the slip came down.

57:49.800 --> 57:57.360
So we're already doing a lot of art pre-production on next-gen titles and other titles.

57:57.400 --> 58:02.840
We even have concept artists being hired to do sketches to try and figure out, you know,

58:02.840 --> 58:07.800
what our look will be and basically just explore, you know, the space of looks.

58:07.800 --> 58:12.560
It turns out to be much faster to have an artist do that than to do a whole bunch of

58:12.560 --> 58:17.040
Maya renders, or even worse, you know, get a game engine up and running and do things

58:17.040 --> 58:18.040
like that.

58:18.040 --> 58:23.760
A big success from Sims 2 art-wise was probably the auto-content build.

58:23.760 --> 58:29.280
Once that was up and running, it worked very smoothly, and given the sheer amount of models

58:29.280 --> 58:35.160
and animations that went through that system, I think it did a good job of getting everything

58:35.160 --> 58:38.320
in the right place at the right time.

58:38.320 --> 58:44.120
Engineering, basically the lesson is just keep it simple.

58:44.120 --> 58:50.440
We way overdid the complexity on Sims 2, and that was pretty, actually pretty much localized

58:50.440 --> 58:51.440
to that team.

58:51.440 --> 58:55.480
I've had experience on other teams at Maxis, and they've been much more practical and much

58:55.480 --> 58:56.880
more focused.

58:56.880 --> 59:00.800
So basically, the simpler you can keep the base-level things, the more complexity you

59:00.800 --> 59:01.800
can build on top of that.

59:01.800 --> 59:07.480
I'm sure I'm not telling anyone anything particularly new here.

59:07.480 --> 59:11.440
And basically, yeah, rapid iteration is, as some of the previous speakers have said, is

59:11.440 --> 59:15.680
absolutely the key to game development.

59:15.680 --> 59:20.600
Technology-wise, we're kind of moving to this commodity rendering system.

59:20.600 --> 59:24.600
As someone said to me, the great thing about it is that it's something that everyone can

59:24.600 --> 59:28.920
hate, kind of together, external.

59:28.920 --> 59:32.680
We're adopting the effect system that kind of worked out pretty well for authoring large

59:32.680 --> 59:39.120
amounts of dynamic content and complicated content, and also as a prototyping tool.

59:39.120 --> 59:44.000
We're using that quite a bit on current pre-production systems to kind of figure out what something

59:44.000 --> 59:46.960
will look like.

59:46.960 --> 59:52.880
We're switching much more to text-based scripting, as I said before, and we had a kind of game

59:52.880 --> 59:59.560
object asset database on Sims 2, but it was kind of, it was a carryover from Sims 1, and

59:59.560 --> 01:00:00.560
it wasn't the best.

01:00:00.560 --> 01:00:08.160
So we actually have a team working on building, you know, a proper kind of production hardware

01:00:08.160 --> 01:00:13.320
system.

01:00:13.320 --> 01:00:15.240
And that's pretty much it.

01:00:15.240 --> 01:00:20.560
These are some of the people at Maxis who were helpful in giving me advice and statistics

01:00:20.560 --> 01:00:21.560
and all the rest of it.

01:00:21.560 --> 01:00:22.560
All right.

01:00:22.560 --> 01:00:23.560
Thank you.

01:00:23.560 --> 01:00:24.560
Thank you.

01:00:24.560 --> 01:00:25.560
So, who's got questions?

01:00:25.560 --> 01:00:26.560
All right.

01:00:26.560 --> 01:00:27.560
Go ahead and start, and I will bring the mic.

01:00:27.560 --> 01:00:28.560
So, I have a couple of questions for you to answer.

01:00:28.560 --> 01:00:57.000
Why was your content building system centralised?

01:00:57.000 --> 01:01:00.840
Why was it important for your artists to build their content on the server rather than for

01:01:00.840 --> 01:01:03.880
them to actually be building the content on their machine?

01:01:03.880 --> 01:01:06.880
Like, why does an artist need to check something in in order to validate it?

01:01:06.880 --> 01:01:07.880
That was something I didn't quite understand.

01:01:07.880 --> 01:01:09.960
They could validate it locally.

01:01:09.960 --> 01:01:14.200
So this is just art that is actually going into the game, you know, into the next build,

01:01:14.200 --> 01:01:15.880
daily build.

01:01:15.880 --> 01:01:19.880
Locally they would be developing this model animation that had a preview button, and it

01:01:19.880 --> 01:01:25.000
would kind of do an instant export then and fire up the remote, the, sorry, local viewer.

01:01:25.000 --> 01:01:29.680
We had a very lightweight kind of viewer application, and basically I'd be able to

01:01:29.680 --> 01:01:30.680
preview it in there.

01:01:30.680 --> 01:01:31.680
That was another sign of actually putting it in the game for a re-build.

01:01:31.680 --> 01:01:32.680
Right.

01:01:32.680 --> 01:01:33.680
You don't need to put it in the game for a re-build if it went through the third person

01:01:33.680 --> 01:01:34.680
and you put it in the middle.

01:01:34.680 --> 01:01:35.680
Yes.

01:01:35.680 --> 01:01:36.680
The content building is the same.

01:01:36.680 --> 01:01:37.680
Yeah.

01:01:37.680 --> 01:01:38.680
Same code.

01:01:38.680 --> 01:01:54.680
At the beginning, there was a lot of talk about the importance of the tech lead, and

01:01:55.360 --> 01:01:57.200
kind of vetting decisions.

01:01:57.200 --> 01:02:04.480
I'm wondering how your, this comm system got built, and your templates got built, and all

01:02:04.480 --> 01:02:09.120
these, how these things lasted more than a week or two in development.

01:02:09.120 --> 01:02:10.120
Yeah.

01:02:10.120 --> 01:02:15.120
I have to be somewhat tactful here.

01:02:15.120 --> 01:02:16.120
You've got to have good tech leads.

01:02:16.120 --> 01:02:17.120
That's what it comes down to.

01:02:17.120 --> 01:02:18.120
These weren't things that were pushed on you by, like, I noticed the outsourcing was

01:02:18.120 --> 01:02:19.120
pushed on you.

01:02:19.120 --> 01:02:20.120
Were these other things pushed on you by EA or other people?

01:02:20.120 --> 01:02:21.120
Right.

01:02:21.120 --> 01:02:22.120
Right.

01:02:22.120 --> 01:02:23.120
That's not the answer.

01:02:23.120 --> 01:02:24.120
Come on.

01:02:24.120 --> 01:02:25.120
You're right.

01:02:25.120 --> 01:02:38.760
So they shouldn't have lasted more than a few weeks.

01:02:38.760 --> 01:02:41.120
I mean, there were obviously bad things at the time.

01:02:41.120 --> 01:02:45.560
Unfortunately, the people who pushed them basically had the support of upper-level engineering

01:02:45.560 --> 01:02:53.640
management who weren't really used to, had not had engineering experience before.

01:02:53.640 --> 01:02:58.320
So you talked about the third-direct scheme for optimizing graphics.

01:02:58.320 --> 01:03:03.160
How much do you think the scene graph was a factor in having to go to that scheme?

01:03:03.160 --> 01:03:07.920
Well, you see, it was, I think it was a large factor.

01:03:07.920 --> 01:03:09.720
It's not the only factor.

01:03:09.720 --> 01:03:13.760
We would have had to do something else on low-end machines, because we were definitely

01:03:13.760 --> 01:03:16.520
up against the batch count.

01:03:16.520 --> 01:03:20.120
But there are a number of other things we could have done there, you know, accumulating

01:03:20.120 --> 01:03:26.520
objects, well, A, better LOD, and B, accumulating objects into multiple buffers, and all sorts

01:03:26.520 --> 01:03:27.520
of things.

01:03:27.520 --> 01:03:33.760
This question may not be relevant, so please feel free not to answer it, but you mentioned

01:03:33.760 --> 01:03:38.040
twice that the toolkit approach, you know, you thought was something really good you

01:03:38.040 --> 01:03:39.040
wanted.

01:03:39.040 --> 01:03:40.040
What exactly is that?

01:03:40.360 --> 01:03:46.600
I mean, rather than providing, like, a top-level interface that dictates how something is going

01:03:46.600 --> 01:03:52.880
to be rendered, which was pretty much what we had, you instead get a bunch of utility

01:03:52.880 --> 01:03:59.560
routines that you're assembling into, you know, a rendering engine.

01:03:59.560 --> 01:04:05.040
So one of our problems was that the interface between, you know, our graphics layer and

01:04:05.040 --> 01:04:08.900
our gameplay layer was far too high.

01:04:08.900 --> 01:04:14.780
So there were a lot of things that, say, we wanted to do in the gameplay arena that

01:04:14.780 --> 01:04:18.700
we couldn't, without actually reaching down to the graphics layer and getting changes

01:04:18.700 --> 01:04:20.140
made.

01:04:20.140 --> 01:04:28.820
And by the same token, which meant we were kind of limited in what we could do.

01:04:28.820 --> 01:04:35.540
I had one other thought, but I forgot that.

01:04:35.540 --> 01:04:39.500
After the slip, was there expectation to up the target that you were going for, rather

01:04:39.500 --> 01:04:46.580
than a DX7 to a DX8 kind of platform, because you were slipping by about seven, eight months?

01:04:46.580 --> 01:04:47.580
Yeah.

01:04:47.580 --> 01:04:48.580
No.

01:04:48.580 --> 01:04:52.980
And I think the reason for that is that we were initially only slipping five or six months,

01:04:52.980 --> 01:04:54.340
and then we slipped again.

01:04:54.340 --> 01:04:57.820
But it's not a slip if no one outside hears about it.

01:04:57.820 --> 01:05:05.180
Yeah, but the pressure wasn't actually on at that point, because at the time where we

01:05:05.180 --> 01:05:10.460
could actually have done anything about this, and I'll point out, we did ship with a DX9

01:05:10.460 --> 01:05:11.460
path.

01:05:11.460 --> 01:05:17.500
The reason that we cut the DX8 path was our system was too inefficient to handle the massive,

01:05:17.500 --> 01:05:21.660
you need a lot of different shaders to be generated under that kind of path, as Chris

01:05:21.660 --> 01:05:26.320
was talking about earlier, and that killed us.

01:05:26.320 --> 01:05:32.380
But at the time, there weren't a lot of titles out there that were even using DX8 cards well

01:05:32.380 --> 01:05:34.060
on the PC.

01:05:34.060 --> 01:05:39.420
Was there some guy whose entire job was to do the DX8 path, and then it got cut, or anything

01:05:39.420 --> 01:05:40.420
like that?

01:05:40.420 --> 01:05:42.100
It wasn't his entire job.

01:05:42.100 --> 01:05:47.180
We did actually, towards the end, say for the last year, we had a guy who was pretty

01:05:47.180 --> 01:05:50.820
much dedicated to material scripting.

01:05:50.820 --> 01:05:55.520
Up to that point, it had been done by me and another guy.

01:05:55.520 --> 01:06:00.380
So yeah, that probably was the majority of his job, because all that stuff had to be

01:06:00.380 --> 01:06:05.620
written in Asm, and a lot of different cases to handle all the different, you know, to

01:06:05.620 --> 01:06:06.620
pack stuff in.

01:06:06.620 --> 01:06:12.380
An interesting stat is that the object we had with the largest number of bones was actually

01:06:12.380 --> 01:06:18.060
the bed, which had 180 bones, and obviously had to be split up for the hardware.

01:06:18.060 --> 01:06:21.900
That's just because there are a lot of different ways to get into a bed, and you know, the

01:06:21.900 --> 01:06:25.940
blankets have to ruffle up, and all the rest of it.

01:06:25.940 --> 01:06:29.420
It's never the objects that you kind of expect that are the most complicated.

01:06:30.220 --> 01:06:35.140
Okay, so I'm going to do Ken, and then we're going to be open now to both further questions,

01:06:35.140 --> 01:06:38.140
and also just if you have statements, or you want to say anything about your own development

01:06:38.140 --> 01:06:40.140
process, we'll kind of pass the mic around.

01:06:40.140 --> 01:06:47.540
So I wanted to ask you about the structure of the engineering team, and how hierarchy

01:06:47.540 --> 01:06:48.540
worked.

01:06:48.540 --> 01:06:54.380
You know, was there someone who sort of ran herd over scheduling, in addition to, you

01:06:54.380 --> 01:06:57.220
know, perhaps multiple programming leads?

01:06:57.220 --> 01:07:00.340
You know, how did that all sort of flesh out?

01:07:00.340 --> 01:07:03.820
So one of our problems was that it changed a lot.

01:07:03.820 --> 01:07:07.060
That was probably actually a problem I didn't touch on on the Simstude, like right throughout

01:07:07.060 --> 01:07:08.060
the process.

01:07:08.060 --> 01:07:12.820
I mean, certainly up until the last couple of years, we were constantly changing producer,

01:07:12.820 --> 01:07:16.740
or whoever was running the team, as they got sucked off to, you know, be thrown on the

01:07:16.740 --> 01:07:19.740
latest fire on other projects.

01:07:19.740 --> 01:07:26.260
So for the last couple of years, we had a development director, and the engineering

01:07:26.300 --> 01:07:31.860
staffing under that, she was directly running the engineering team, like most of the engineering

01:07:31.860 --> 01:07:35.260
team, and then the leads were kind of over here.

01:07:35.260 --> 01:07:38.580
And I think one of the things that kind of went wrong is that the leads had a bit too

01:07:38.580 --> 01:07:43.420
much autonomy, and a bit too much separation from the rest of the engineering team.

01:07:43.420 --> 01:07:47.580
So you think, in your perfect world, would that not happen again?

01:07:47.900 --> 01:07:58.700
Yeah, in my perfect world, we would have a dedicated engineering manager, and everyone

01:07:58.700 --> 01:07:59.700
would be below that.

01:07:59.700 --> 01:08:00.700
How many leads?

01:08:00.700 --> 01:08:03.540
I mean, a lot of it just depends on the people you have involved.

01:08:03.540 --> 01:08:10.420
Like the previous title, SimCity 4, we had basically one guy who was pretty much the

01:08:10.420 --> 01:08:15.500
tech lead, and he was highly competent, and everything pretty much just worked.

01:08:16.500 --> 01:08:18.500
How many leads were there?

01:08:18.500 --> 01:08:19.500
On Sims 2?

01:08:19.500 --> 01:08:20.500
Yeah.

01:08:20.500 --> 01:08:22.500
It depends how you classify it.

01:08:22.500 --> 01:08:24.500
There was a graphics lead.

01:08:24.500 --> 01:08:26.500
Am I just asking the wrong question here?

01:08:26.500 --> 01:08:31.500
There was a graphics lead, there was an animation lead, there was kind of a lead engineer who

01:08:31.500 --> 01:08:36.500
covered everything else, but tended to ping-pong into management and back as we went through,

01:08:36.500 --> 01:08:37.500
like, headcount changes.

01:08:37.500 --> 01:08:41.500
And so, you know, didn't really get heavily involved.

01:08:42.500 --> 01:08:44.500
What if I used you as a poet, maybe?

01:08:44.500 --> 01:08:45.500
I'm confused.

01:08:45.500 --> 01:08:46.500
I don't know.

01:08:50.500 --> 01:08:54.500
We do have a whiteboard, if you want to do that.

01:08:54.500 --> 01:08:56.500
We understand if you don't, but, you know.

01:08:56.500 --> 01:08:59.500
So, you're after, like, the engineering structure?

01:08:59.500 --> 01:09:00.500
Yeah.

01:09:00.500 --> 01:09:01.500
Yeah.

01:09:01.500 --> 01:09:02.500
Okay, so towards the end...

01:09:02.500 --> 01:09:05.500
It's longer than time, towards the end, right?

01:09:05.500 --> 01:09:07.500
Okay, so moment one, towards the end.

01:09:07.500 --> 01:09:13.500
It was basically DD, graphics, animation, and, like, other lead.

01:09:15.500 --> 01:09:16.500
Development director.

01:09:16.500 --> 01:09:21.500
Yeah, so another problem we had then was that our development director, because we were

01:09:21.500 --> 01:09:26.500
doing a lot of this massive CM stuff for the first time, a lot of the attention was kind

01:09:26.500 --> 01:09:29.500
of diverted onto CM and managing the daily build.

01:09:29.500 --> 01:09:33.500
So there wasn't always a lot of attention left over for the engineering process.

01:09:33.500 --> 01:09:36.500
So DD leads...

01:09:37.500 --> 01:09:40.500
I mean, the problem here is just that it was confused.

01:09:40.500 --> 01:09:46.500
Like, I can draw a picture at one stage which basically had most of the engineers under

01:09:46.500 --> 01:09:48.500
this third lead.

01:09:49.500 --> 01:09:50.500
Where are we?

01:09:51.500 --> 01:09:53.500
Graphics, animation, simulation.

01:09:54.500 --> 01:09:55.500
Um...

01:09:59.500 --> 01:10:04.500
So, like, this was just before the slip, I think.

01:10:04.500 --> 01:10:08.500
After the slip, you know, it was obvious that DD was overburdened, so we had more, like,

01:10:08.500 --> 01:10:13.500
three engineering managers who came in and basically each took a bit of the team down

01:10:13.500 --> 01:10:14.500
here.

01:10:16.500 --> 01:10:19.500
And then unfortunately we lost one of them six months later.

01:10:20.500 --> 01:10:21.500
Um...

01:10:21.500 --> 01:10:25.500
Did people sort of migrate over to the graphics and animation leads?

01:10:25.500 --> 01:10:28.500
Did those people actually manage, or were they really just sort of pioneering their

01:10:28.500 --> 01:10:29.500
own?

01:10:29.500 --> 01:10:30.500
So...

01:10:31.500 --> 01:10:32.500
Um...

01:10:34.500 --> 01:10:37.500
So there's manage and then there's task assignment.

01:10:38.500 --> 01:10:39.500
Yeah.

01:10:39.500 --> 01:10:40.500
So basically...

01:10:40.500 --> 01:10:42.500
That's the thing I'm asking about.

01:10:42.500 --> 01:10:46.500
Okay, so task assignment was basically these guys talking to the DD in some cases, let's

01:10:46.500 --> 01:10:48.500
say specific graphics tasks.

01:10:48.500 --> 01:10:49.500
So normal mapping.

01:10:49.500 --> 01:10:54.500
You know, with, say, this guy talking to the DD and then assigning tasks to junior programmers.

01:10:54.500 --> 01:11:00.500
With, say, this guy then asking to see, like, a specification of an interface or things

01:11:00.500 --> 01:11:01.500
like that.

01:11:02.500 --> 01:11:04.500
Animation was very nicely compartmentalized, actually.

01:11:04.500 --> 01:11:11.500
Basically, the reason the animation lead didn't have a lot of direct reports is that he managed

01:11:11.500 --> 01:11:15.500
to maintain that system pretty much all the way through in close conjunction with, you

01:11:15.500 --> 01:11:17.500
know, the technical art director.

01:11:18.500 --> 01:11:22.500
And only at the end had to work with another engineer to implement animation compression

01:11:22.500 --> 01:11:24.500
and all sorts of things like that.

01:11:26.500 --> 01:11:32.500
So I think a lot of people under the simulation guy, they were really being tapped out by

01:11:32.500 --> 01:11:33.500
the development director?

01:11:33.500 --> 01:11:34.500
Yes.

01:11:34.500 --> 01:11:35.500
Okay.

01:11:35.500 --> 01:11:36.500
And so...

01:11:38.500 --> 01:11:39.500
For a while.

01:11:39.500 --> 01:11:40.500
Who...

01:11:41.500 --> 01:11:47.500
The answer to this question is not applicable, but, you know, who enforced policy standards

01:11:47.500 --> 01:11:48.500
across the team?

01:11:49.500 --> 01:11:50.500
Code policy standards or...

01:11:50.500 --> 01:11:51.500
Yeah, policy standards.

01:11:51.500 --> 01:11:56.500
Like, you will deallocate your freaking memory allocation when you clean up your...

01:11:57.500 --> 01:12:02.500
Initially, when the team was smaller, the graphics lead and the simulation lead did.

01:12:03.500 --> 01:12:08.500
As the team grew larger and it became obvious that some of these things were bad ideas,

01:12:09.500 --> 01:12:11.500
everything became much more freeform.

01:12:12.500 --> 01:12:14.500
I'm not sure I can kind of get into too much detail.

01:12:14.500 --> 01:12:15.500
Yeah, back here.

01:12:19.500 --> 01:12:21.500
So while I'm waiting for the microphone on top...

01:12:22.500 --> 01:12:23.500
Go ahead.

01:12:23.500 --> 01:12:32.500
So this is sort of a tangent, but something we're trying is a thing called team software process.

01:12:33.500 --> 01:12:36.500
Carnegie Mellon Software Engineering Institute does it.

01:12:36.500 --> 01:12:39.500
I don't know if anyone's heard of it, but we ran into a project.

01:12:40.500 --> 01:12:46.500
I mean, it wasn't nearly as big as Sims, but where it was kind of crazy and out of control.

01:12:47.500 --> 01:12:49.500
And after that, we wanted to figure out, you know, how could we make it better?

01:12:50.500 --> 01:12:56.500
And so we tried this thing called team software process where sort of all the team members

01:12:57.500 --> 01:13:00.500
are responsible for coming up with the schedule themselves.

01:13:01.500 --> 01:13:03.500
And they're responsible for tracking their own time.

01:13:04.500 --> 01:13:06.500
And there's sort of time set up for reviews and everything.

01:13:07.500 --> 01:13:14.500
And there's a lot of cross-team communication, which seemed to help with a lot of these sort of areas

01:13:14.500 --> 01:13:16.500
where you don't know who's responsible for what.

01:13:17.500 --> 01:13:21.500
And you don't know if the communication is going across the team or back up or stuff like that.

01:13:22.500 --> 01:13:23.500
I don't know.

01:13:23.500 --> 01:13:27.500
So part of the problem here is that this is kind of a chicken and the egg situation.

01:13:28.500 --> 01:13:33.500
There was a lot of churn here just because it was such a high pressure product.

01:13:36.500 --> 01:13:37.500
And as a result, we had a lot of change.

01:13:38.500 --> 01:13:39.500
You know, if something wasn't working, it would be changed.

01:13:40.500 --> 01:13:41.500
We lost people too.

01:13:44.500 --> 01:13:47.500
It might do me better to kind of diagram the situation on SimCity,

01:13:48.500 --> 01:13:49.500
which had almost as many engineers.

01:13:50.500 --> 01:13:53.500
We had like 22 engineers in SimCity at the end, as opposed to about 28 on Sims 2.

01:13:54.500 --> 01:13:57.500
Where we just had a very competent kind of...

01:14:01.500 --> 01:14:04.500
We called them a DD, but at that point it was like the Maxis DD,

01:14:05.500 --> 01:14:07.500
which is kind of more of an engineering manager.

01:14:08.500 --> 01:14:09.500
We had a tech lead.

01:14:10.500 --> 01:14:14.500
And then basically all the engineers were being managed by the engineering manager

01:14:15.500 --> 01:14:19.500
and being given a large amount of technical direction by the tech lead.

01:14:20.500 --> 01:14:21.500
But not too much.

01:14:22.500 --> 01:14:26.500
I think one of the problems with Sims 2 was actually there was too much technical direction.

01:14:29.500 --> 01:14:30.500
There was too much.

01:14:31.500 --> 01:14:33.500
Yeah, too much processes is what I'm trying to say.

01:14:34.500 --> 01:14:36.500
Yeah, just to follow up with what we're doing,

01:14:37.500 --> 01:14:40.500
it's something similar to what you described, and it's called Scrum.

01:14:41.500 --> 01:14:44.500
And it's more of an agile methodology where a team's self-organized.

01:14:45.500 --> 01:14:46.500
They break down their own tasks.

01:14:46.500 --> 01:14:47.500
They create their own estimates.

01:14:48.500 --> 01:14:53.500
And then they essentially meet on a daily basis to re-estimate remaining tasks

01:14:54.500 --> 01:14:59.500
or basically burn down on this monthly iteration that they do.

01:15:00.500 --> 01:15:05.500
And it does remove a lot of burden from lead programmers, managers,

01:15:06.500 --> 01:15:10.500
for having to break down these tasks and estimate them for other people.

01:15:11.500 --> 01:15:12.500
Right.

01:15:13.500 --> 01:15:16.500
Yeah, so the blob wasn't quite as ill-defined as I'm making out here.

01:15:17.500 --> 01:15:18.500
You're reminding me.

01:15:19.500 --> 01:15:22.500
You know, we had teams within that who their function was well understood.

01:15:23.500 --> 01:15:25.500
You know, we had some people working on a particular section of,

01:15:26.500 --> 01:15:27.500
say, the dirty rec system.

01:15:28.500 --> 01:15:29.500
Others on effects and lighting.

01:15:30.500 --> 01:15:32.500
Others on the simulation routing.

01:15:34.500 --> 01:15:38.500
I'm just unable to give a very clear answer to, like, the overall structure.

01:15:38.500 --> 01:15:40.500
Just because it changed recently, often.

01:15:41.500 --> 01:15:42.500
Two kind of silly questions, I guess.

01:15:43.500 --> 01:15:44.500
Feel free to ignore them.

01:15:45.500 --> 01:15:48.500
Like, one, could you quantify why the product slipped at one level

01:15:49.500 --> 01:15:50.500
of more granularity?

01:15:51.500 --> 01:15:52.500
Like, it was design.

01:15:53.500 --> 01:15:54.500
It was code.

01:15:55.500 --> 01:15:56.500
The scene graph screwed us.

01:15:57.500 --> 01:15:59.500
I mean, was there, like, is there any way to drill down that one more level

01:16:00.500 --> 01:16:01.500
to kind of, like, place more blame on the slip?

01:16:02.500 --> 01:16:04.500
During the talk, it was pretty much 50% design, 50% engineering.

01:16:04.500 --> 01:16:06.500
The engineering didn't get as much play because it was so obvious.

01:16:07.500 --> 01:16:09.500
You know, the executive producers were mostly concerned with the design.

01:16:10.500 --> 01:16:11.500
They all had input into that.

01:16:12.500 --> 01:16:15.500
But the fact was also we couldn't have made engineering by a considerable amount.

01:16:16.500 --> 01:16:19.500
The second kind of silly question is, like, what's the feedback loop here,

01:16:20.500 --> 01:16:21.500
like, corporate-wise?

01:16:22.500 --> 01:16:23.500
I mean, the game came out and beat Forecast.

01:16:24.500 --> 01:16:25.500
It did slip, which cost money.

01:16:26.500 --> 01:16:28.500
But, like, you know, kind of screwed up project, yet,

01:16:29.500 --> 01:16:30.500
you know, we're still making it.

01:16:30.500 --> 01:16:32.500
So, like, where's the, like, where do we, when does,

01:16:33.500 --> 01:16:34.500
and this is kind of a meta question, like, when does process

01:16:35.500 --> 01:16:36.500
actually end up screwing us?

01:16:37.500 --> 01:16:39.500
Like, in this case, it costs EA, you know, 20 million more dollars,

01:16:40.500 --> 01:16:42.500
but the game's still going to make $700 million or something insane like that.

01:16:43.500 --> 01:16:44.500
So, like.

01:16:44.500 --> 01:16:45.500
Right.

01:16:45.500 --> 01:16:46.500
I actually had a bullet point on one of my slides at some point

01:16:47.500 --> 01:16:48.500
kind of saying, don't take the wrong listen from this.

01:16:49.500 --> 01:16:50.500
Because one of the listens is, too, is kind of,

01:16:51.500 --> 01:16:53.500
you can screw up a project, but you can't screw up a game.

01:16:54.500 --> 01:16:56.500
So, like, when does, and this is kind of a meta question,

01:16:57.500 --> 01:16:58.500
like, when does process actually end up screwing us?

01:16:58.500 --> 01:16:59.500
Because one of the listens is, too, is kind of,

01:17:00.500 --> 01:17:01.500
you can screw up and survive.

01:17:02.500 --> 01:17:03.500
And I don't think we can screw up to this extent

01:17:04.500 --> 01:17:05.500
and continue to survive.

01:17:06.500 --> 01:17:11.500
So, that's why I'm also saying we have to improve our process.

01:17:12.500 --> 01:17:13.500
Yeah, I was just going to say, one of the things you mentioned,

01:17:14.500 --> 01:17:16.500
and there was a question over here about when you're slipping,

01:17:17.500 --> 01:17:19.500
did they make you think of other graphics cards?

01:17:20.500 --> 01:17:21.500
And I know every publisher I've ever dealt with,

01:17:22.500 --> 01:17:23.500
they say, you can slip, but you've got to get me

01:17:24.500 --> 01:17:27.500
my seven pet peeve features that you weren't going to give me before.

01:17:28.500 --> 01:17:31.500
And that always ends up almost costing me more than the slips.

01:17:32.500 --> 01:17:33.500
So, we were aware of that problem, yeah.

01:17:34.500 --> 01:17:35.500
And we were, at least on the engineering side,

01:17:36.500 --> 01:17:38.500
very careful not to add, basically,

01:17:39.500 --> 01:17:40.500
every new engineering feature that was going to be added

01:17:41.500 --> 01:17:43.500
during the slip had to go through a pretty formal process,

01:17:44.500 --> 01:17:46.500
like a natural TDD and all the rest of it.

01:17:47.500 --> 01:17:48.500
Yeah, because otherwise you could just get through the month.

01:17:49.500 --> 01:17:50.500
Yes, yes.

01:17:51.500 --> 01:17:53.500
There's also that infamous mythical man month thing about,

01:17:54.500 --> 01:17:56.500
if you're going to slip, you know, slip a long time.

01:17:57.500 --> 01:17:59.500
Don't do incremental slips and just...

01:18:00.500 --> 01:18:02.500
You know, the other bad thing about slipping is just,

01:18:03.500 --> 01:18:04.500
you've had people who were crunching for, say,

01:18:05.500 --> 01:18:07.500
four or five months, doing six or seven day weeks,

01:18:08.500 --> 01:18:10.500
and extending that for another six months.

01:18:12.500 --> 01:18:14.500
So, I was just going to comment on something Chris said

01:18:15.500 --> 01:18:17.500
about the, like, when is the process thing actually going to hurt,

01:18:18.500 --> 01:18:19.500
which is just that I think that it's pretty clear

01:18:20.500 --> 01:18:21.500
that a lot of times the process thing,

01:18:22.500 --> 01:18:23.500
when things slip long and that sort of stuff,

01:18:23.500 --> 01:18:25.500
it doesn't actually hurt because there's not, like,

01:18:26.500 --> 01:18:28.500
sufficient competition in the games, you know,

01:18:29.500 --> 01:18:31.500
in the games that are out there to make that actually hurt you.

01:18:32.500 --> 01:18:33.500
You know, I mean, if a game like The Sims 2 slips for a year

01:18:34.500 --> 01:18:35.500
or two years, it doesn't really matter

01:18:36.500 --> 01:18:38.500
because there aren't, like, competitors coming in

01:18:39.500 --> 01:18:40.500
and selling to that market aggressively

01:18:41.500 --> 01:18:42.500
in a way that takes away from it.

01:18:43.500 --> 01:18:44.500
I mean, that's usually when companies fail

01:18:45.500 --> 01:18:46.500
because of process things.

01:18:47.500 --> 01:18:48.500
It's because someone comes and, you know,

01:18:49.500 --> 01:18:50.500
takes advantage of that.

01:18:51.500 --> 01:18:52.500
So, I would actually slightly disagree with that.

01:18:53.500 --> 01:18:54.500
As the games get bigger and bigger,

01:18:55.500 --> 01:18:57.500
the cost increases for, like, a number of reasons.

01:18:58.500 --> 01:18:59.500
One is just marketing.

01:19:00.500 --> 01:19:01.500
So, we lost a lot of money by not hitting that date

01:19:02.500 --> 01:19:03.500
just because we already had, like, TV ads booked,

01:19:04.500 --> 01:19:05.500
you know, various things like that.

01:19:06.500 --> 01:19:07.500
The other is burn rate.

01:19:08.500 --> 01:19:09.500
You know, the burn rate for slipping two years

01:19:10.500 --> 01:19:13.500
is phenomenal with a big project like that.

01:19:18.500 --> 01:19:19.500
I was going to ask a question

01:19:20.500 --> 01:19:21.500
which might lead to a more general discussion,

01:19:21.500 --> 01:19:23.500
which is, can you talk about the background

01:19:24.500 --> 01:19:25.500
that your object engineers had?

01:19:26.500 --> 01:19:27.500
Like, you refer to them as object engineers,

01:19:28.500 --> 01:19:29.500
which I guess means that they are trained in engineering,

01:19:30.500 --> 01:19:31.500
but at the same time,

01:19:32.500 --> 01:19:33.500
they're exclusively using a visual tool

01:19:34.500 --> 01:19:35.500
and getting frustrated with it

01:19:36.500 --> 01:19:37.500
because they'd rather be using command line text-based things.

01:19:38.500 --> 01:19:40.500
So, kind of, what is their background?

01:19:41.500 --> 01:19:42.500
Were they engineers and designers?

01:19:43.500 --> 01:19:44.500
Would you rather have had engineers?

01:19:45.500 --> 01:19:46.500
Would you rather have had designers?

01:19:47.500 --> 01:19:48.500
And how would you change the tools

01:19:49.500 --> 01:19:50.500
just in a general sense to tailor to that?

01:19:51.500 --> 01:19:52.500
So, they weren't, you know,

01:19:53.500 --> 01:19:54.500
they weren't C++ engineers.

01:19:55.500 --> 01:19:56.500
They were, although some of them,

01:19:57.500 --> 01:19:58.500
you know, had that capability,

01:19:59.500 --> 01:20:00.500
they were gameplay scripters.

01:20:01.500 --> 01:20:02.500
They had to basically,

01:20:03.500 --> 01:20:04.500
the way I tend to think about them is just,

01:20:05.500 --> 01:20:06.500
you know, I've had some experience

01:20:07.500 --> 01:20:08.500
in the animation industry

01:20:09.500 --> 01:20:10.500
and that kind of, that TD-level glue

01:20:11.500 --> 01:20:12.500
that holds the whole process together.

01:20:13.500 --> 01:20:14.500
So, you know, they're taking the assets,

01:20:15.500 --> 01:20:16.500
you know, there's the engine

01:20:17.500 --> 01:20:18.500
that's being supported,

01:20:18.500 --> 01:20:19.500
the engineers,

01:20:20.500 --> 01:20:21.500
and together with the producers

01:20:22.500 --> 01:20:23.500
knitting it all together

01:20:24.500 --> 01:20:25.500
to make the game out of it.

01:20:26.500 --> 01:20:27.500
In a game like The Sims,

01:20:28.500 --> 01:20:29.500
where, you know,

01:20:30.500 --> 01:20:31.500
the gameplay is nicely compartmentalized like that,

01:20:32.500 --> 01:20:33.500
you know, the kind of combination

01:20:34.500 --> 01:20:35.500
level designer,

01:20:36.500 --> 01:20:37.500
yeah.

01:20:38.500 --> 01:20:39.500
Yeah, basically.

01:20:40.500 --> 01:20:41.500
All right.

01:20:42.500 --> 01:20:43.500
Well, thank you, Andrew.

01:20:44.500 --> 01:20:45.500
Thank you.

01:20:45.500 --> 01:20:46.500
Thank you.

01:20:47.500 --> 01:20:48.500
Thank you.