I ran into an interesting bug this week.

Lets set the scene. We have a website where admin users can upload a SCORM file with some extra details and it will add it to the website for people to view.

When the file is uploaded, we use SharpZipLib to have a look inside the zip file and confirm the contents to make sure it's a valid SCORM file. Then we put all the files where they need to be. It's been running for a couple of months but then suddenly the client can no longer upload files. Weird, right? The error messsage told us that the SCORM file is not valid.

Debugging

We started the debugging process by asking for a copy of the file that was failing and looking at the process logs. There were no obvious failures or reasons for it to fail while debugging the uploading of the file using our local machines. However, good news - the production logging turns up an error we can start investigating with:

Compressed size mismatch between central header(743) and local header(0) Server stack trace: at ICSharpCode.SharpZipLib.Zip.ZipFile.TestLocalHeader

Well I wasn't expecting a header mismatch - we are going to need to take a closer look at these zip files.

Can you spot the oddity with this? Why would it be a different file size? Both 'Uploaded' files have been uploaded using chrome on Windows using the same page.

Unzipping both files and then comparing them returns an exact match so nothing has happened to the files.

Better have a closer look at these zip files (credit to Wikipedia for the header data) -

Well that explains the error message above. The compressed size has been wiped out, something during the process is modifying the headers of the zip file but not the contents of the file.

I'm starting to get a feeling we have something interfering with the upload rather than a error with the service let's have a more user friendly look at that header data using my good friend 7-Zip.

Well that has pretty much confirmed a UTM style device at the client's location is unzipping then pressumabilty scanning the contents of the zip file, creating a new zip file using its own settings but not recalculating the compressed header size for each of the files. Presumably this is because its doing it on the fly and wants to be as quick as possible.

Problem found, what is the way forward?

Lets have a look at the SharpZipLib to see if there is anything we can do to ignore the header checks.

The checks are done in this function:

long TestLocalHeader(ZipEntry entry, HeaderTest tests)

Headers are read out byte by byte:

var extractVersion = (short)(ReadLEUshort() & 0x00ff);
var localFlags = (short)ReadLEUshort();
var compressionMethod = (short)ReadLEUshort();
var fileTime = (short)ReadLEUshort();
var fileDate = (short)ReadLEUshort();
uint crcValue = ReadLEUint();
long compressedSize = ReadLEUint();
long size = ReadLEUint();
int storedNameLength = ReadLEUshort();
int extraDataLength = ReadLEUshort();

And the test itself:

// Tests that apply to both data and header.

// Size can be verified only if it is known in the local header.
// it will always be known in the central header.
if (((localFlags & (int)GeneralBitFlags.Descriptor) == 0) ||
	((size > 0 || compressedSize > 0) && entry.Size > 0)) {

	if ((size != 0)
		&& (size != entry.Size)) {
		throw new ZipException(
            string.Format("Size mismatch between central header({0}) and local header({1})",
                entry.Size, size));
	}

	if ((compressedSize != 0)
		&& (compressedSize != entry.CompressedSize && compressedSize != 0xFFFFFFFF && compressedSize != -1)) {
		throw new ZipException(
            string.Format("Compressed size mismatch between central header({0}) and local header({1})",
            entry.CompressedSize, compressedSize));
	}
}

Not ideal! That leaves us with two issues-

1. No overrides or options to skip that test within SharpZipLib mixed with the library throwing a hard error.
2. We have no idea if the client will be able to resolve the issue with the hardware.

Solution

We decided to have a quick look at alternatives before going down the route of forking the SharpZipLib and adding an override. Luckily, a little bit of research and testing later it's System.IO.Compression.ZipArchive to the rescue! It's now all back up and running and able to handle the slightly dodgy zip files being uploaded, sorted!