Sunday, 31 July 2016

Revisiting Activision's Interstate 76


Interstate 76 – A blast from the past

Even for me this is a bit of a pointless exercise, but....


Years ago I used to play Interstate 76; a car combat game set in the 70's. Think classic American muscle cars, add guns, rockets and armour with a funk soundtrack and you're not far off.

The graphics engine was pretty ropey, even by the standards of the time, a re-purposed version of the Mechwarrior engine it was low poly and full of bugs, but the thing I76 had was style – the soundtrack, the voice acting, the single player storyline and cut scenes all fit together to give the game as a whole a wonderful feel. It was the kind of game you'd play over again and again just to listen to the soundtrack while driving around, and watch the cut-scenes play out.

Anyhow, digging around through some old disks I came across the original game, and decided to take a look at how the game data is packed on disk.

I told you this was pointless.

Background Reading

Most of the i76 stuff has dropped off the web, but the wayback machine has archived a few sites, and there's a couple of useful general links

Some initial references:
Hoppo's Custom Paint Job Guide (via the wayback machine)

Xentax formats information

A vehicle hacks site, from a geocities mirror. I never thought I'd miss geocities, but....


Also videoventure has information on Battlezone, which shares some engine details with I76 – useful for picking apart the geometry & map files

And similarly this github page has some code which gave me useful information on the geometry format

And since GoG re-released the original there seem to be some active games still running on
 

Top Level Stuff

The bulk of the game data is in two files:
  • I76.ZFS
  • DATABASE.MW2
These files are themselves containers for multiple files. I'm dealing with the original I-76 disks, however I've read that the I76 expansion pack (Nitro Riders) version of the ZFS file may use LZO compression – I'd have to dig out my copy to verify that though.
 
Data is stored in little endian format, so to read out a  32 byte integer we can use code like:

uint32_t getuInt32()
{
QByteArray ba = ReadData(4);
uint32_t v;
  uint8_t a,b,c,d;
   a = ba.at(3);
   b = ba.at(2);
   c = ba.at(1);
   d = ba.at(0);
  v = (a << 24) | (b << 16) | (c << 8) | d;
return v;
}
Taking each file in turn:

ZFS file


The file is divided into a number of “directory” sections, each of which contains a collection of files.

Each directory has a directory header which details the offset to the next directory header (the last directory entry has a '0' for this value).

Each directory header is followed by a number of file headers, each of which details the name of a file, the offset inside the ZFS and the size of the data. 
Following the file headers is the data for the files themselves, which continues up until the next directory header.

The file opens with the archive header, which details the version of the file and the number of files “per directory” as well as the total number of files in the archive.

So the layout is:
 
  Breaking down the individual sections further, the archive header has the following layout:


Size
Meaning
4 Byte Magic Ident (ZFSF)
uint32 version number
uint32 unknown
uint32 number of files in each directory
uint32 number of files in total
uint32 always NULL
uint32 unknown


And so we can parse this out from the beginning of the file by:

  ar.ident = getString(4);
  ar.version = getuInt32();
  ar.unknown_16 = getuInt32();
  ar.files_per_directory = getuInt32();
  ar.total_files = getuInt32();
  ar.null_marker = getuInt32();
  ar.unknown_28 = getuInt32();

Following this we know the number of files in each directory entry and the total number of files in the archive.

Immediately after this we find the first directory header – we can simply extract the next directory offset with

d.next_offset= getuInt32();

And then we pull out the file headers, based on the files_per_directory value we extracted from the archive header.

Each file header has the structure

Size
Meaning
16 Byte name
uint32 Offset to this file data (i.e. where in the datablock)
uint32 id
uint32 length
uint32 unknown
uint32 always NULL


And can be parsed with code like:
  fh.name = getString(16);
  fh.offset = getuInt32();
  fh.id = getuInt32();
  fh.length = getuInt32();
  fh.unknown_3 = getuInt32();
  fh.null = getuInt32();



When we get to the end of the file (and run out of files to process) we'll start to pull out null data in these file header values, but the headers themselves will always be present.

So when we have pulled out the number of file header that files_per_directory count tells us to expect then we're at the data block and can start to load the file data using the offset and length details, and save it to the name given.

When all the files here have been loaded we can pick up the next directory from the next offset value we got from the directory header and repeat the process, until the next directory offset is zero.


MW2 File


This is considerably simpler. The opening 32 bit value from the file gives the count of entries in this file, and then for each file there follows the offset value to the data start.

Once collected then we can simply dump out each section of the MW2 file to an individual output file. In this case we simply have offset numbers, rather than names.

From the .MW2 file we get a few obvious audio clips from the in game menu operations as well as a bunch of alternate data, but the bulk of interesting data is in the ZFS.

What we get


We get a collection of audio clips, maps, car data, credits, etc.

Taking these in turn 

Audio Files


.wav: Audio clips – radio chatter, in game dialog and effects, Taurus' poems, other stuff. 

.gpw : Prefixed wav files. These are various in game effect noises with 28 bytes of prefix starting “GAS0” and padded with 0xff.


More Packed archive files

The ZFS contains yet more packed data files: ".pak" - really there are two files here, the index and the data file.

.pix: these describe the contents of each .pak in terms of file names & data offset

.pak: additional packed data – multiple files as indicated in the .pix


BWD2 Files

A number of other files are prefixed “BWD2” and have a standard 28 byte file header, which includes a character file type (up to 8 characters, of which the first three characters generally match the extensions):

File Extension
Purpose
.cdf
Misc component definitions
.gdf
Per weapon definitions
.wdf
Wheel type definition
.vdf
Vehicle definitions
.vcf
Vehicle Configuration File
.sdf
Scrounge Texture
.vtf
Vehicle paint (texture) file
.xdf
???



Everything Else

There are also these other files

.act
Palette (affects sky & surface)
.map
Can be either a Sky or Surface texture or Level map (mission maps)
Opens with a pair of 32 bit dimension numbers, representing the size of the image followed by the image map.
.fsi
text file; a list of car setups (one per trip episode ??)
.lum
Luma table
.tbl
Translucency table
.cbk
Colour Bank
.geo
Geometry File, describing vertices, normals and texture mapping
.msk
?? Four files which match available resolutions ??
.rtm
Map file lists ??
.tmt
Vehicle Image Textures. For the naming convention see Hoppo's page.
.npt
Objective file: Various mission text strings related to objectives
.vqm
Image files
Opens with a pair of 32 bit dimension numbers (e.g. for 32K this is 640*480)
Then a 16 character name string “.CBK” - the colour index


That Geometry file

 
Although there isn't much information on this file format directly it's quite similar to several other Activision game data formats of the era, and specifically public information about “Battlezone 1” formats give us enough clues to pull out the geometry. Obviously there's some guesswork here though...

The initial layout of the file header is 

Size
Meaning
4 Char Magic Number (“GEO.”)
uint32 ??
16 char Name
uint32 Vertex Count
uint32 Face Count
uint32 ??


Next we have all the vertex values. We have 3x4 byte float values (X, Y & Z) per vertex, so this section of the file is 3 x 4 x “Vertex Count” bytes long

Next we have another set of 3x4 byte float values which are the vertex normals – again this covers 3 x 4 x “Vertex Count” bytes

Finally we have the faces. For each available face we have a face header of:

Size
Meaning
uint32 index
uint32 Face vertex count
byte Red
byte Green
byte Blue
(4 byte) float Surface normal X
(4 byte) float Surface normal Y
(4 byte) float Surface normal Z
(4 byte) float Surface normal W (?)
uint32 ??
byte Surface Flags ??
byte Surface Flags ??
byte Surface Flags ??
13 char Texture name for this object (Can be all NULL)
uint32 ??
uint32 ??

Then for each vertex in this face there are four 32bit fields. The first two are the Vertex and Normal reference, and the next two are float values for U & V mapping.

So, with a quick piece of hacky coding we can dump out the vertices, one XYZ triplet per line prefixed with 'v', and the faces, one face per line prefixed with 'f' (and incrementing the vertex refs so they run from '1' rather than '0') and the resulting output is something we can push into blender as a wavefront OBJ import.


And from “PP31BDYM.geo” we get the main body of the Picard Piranha in 229 verts, although blender reckons it it can actually drop 46 verts as duplicates.

 
 

And trying a couple of other models....

Slapping on the extracted textures and putting the various car pieces from the vdf and vcf files together to get a complete model is left as an exercise for the reader...

And just to make me feel old – I played this back in 1997, where the setting of 1976 seemed like ancient history. However from 1997 to 2016 is not far off that 21 year gap...

(Note: edited post to remove some.vqm file speculation that shouldn't have been there)