Tag Archives: macports

SharpZipLib and Mac OS X

TL;DR When creating zip archives with SharpZipLib disable Zip64 format if you care about Mac compatibility.

Extra TL;DR [When creating zip archives with SharpZipLib make sure you set the file size][redux], disabling Zip64 is neither here nor there.

A project I am working on sources a zip archive from a Web service, extracts the XML file from the zip and then does silly amounts of processing of the data in that XML to produce a new XML file which is returned to the user.

But the sourced zip archive cannot be opened using [Python’s zipfile module][zipfile], and when saved on a Mac the archive cannot be opened using the built-in Archive Utility.app. If one double-clicks the zip, Archive Utility.app just compresses it again and sticks “.cpgz” on the end of the file name.

Fortunately the developer of the Web service is very helpful (the service is written in C# and runs on Windows) and although he didn’t know why Macs were having problems (the built-in Windows zip tool can handle the archive fine) he showed me the code that creates the zip file.

Turns out they were using [SharpZipLib, an open-source library for C#][sharpziplib]. And it turns out SharpZipLib creates archives using [Zip64 format][zip64] by default.

The fix was to disable Zip64 when creating the archive. Here’s a trivial command-line program that creates a zip and disables Zip64 for Mac compatibility:

using System;
using System.IO;
using ICSharpCode.SharpZipLib.Core;
using ICSharpCode.SharpZipLib.Zip;

public class ZipTool
{
public static void Main(string[] args)
{
if (args.Length != 2) {
Console.WriteLine(“Usage: ziptool “);
return;
}

using (ZipOutputStream zipout = new ZipOutputStream(File.Create(args[1]))) {
byte[] buffer = new byte[4096];
string filename = args[0];

zipout.SetLevel(9);

// Disable Zip64 for Mac compatibility
zipout.UseZip64 = UseZip64.Off;

ZipEntry entry = new ZipEntry(Path.GetFileName(filename));
entry.DateTime = File.GetLastWriteTime(filename);
zipout.PutNextEntry(entry);

using (FileStream fs = File.OpenRead(filename)) {
int sourceBytes;
do {
sourceBytes = fs.Read(buffer, 0, buffer.Length);
zipout.Write(buffer, 0, sourceBytes);
} while (sourceBytes > 0);
}

zipout.Finish();
zipout.Close();
}
}
}

The disadvantage of disabling Zip64 is you cannot create archives larger than 4 gigabytes, nor can you add files larger than 4 gigabytes before compression.

The advantage of disabling Zip64 is you make me and all my Mac-using hippy friends happy. In concrete terms, disabling Zip64 makes it more likely I will buy you a drink.

Hi Gaston!

Also many thanks to the maintainer of NAnt in [MacPorts][macports], who responded to [my bug report][bug] and pushed an updated NAnt to MacPorts extremely quickly. Although SharpZipLib doesn’t officially support [Mono][mono], it builds without a hitch using the “build-net-2.0” target if you hack the `SharpZlib.build` NAnt script like so:




Also thanks to the Mono project! Saved me having to fire up a Windows virtual machine to figure out this problem.

[zipfile]: http://docs.python.org/library/zipfile.html
[sharpziplib]: http://www.icsharpcode.net/opensource/sharpziplib/
[zip64]: http://en.wikipedia.org/wiki/ZIP_(file_format)#ZIP64
[macports]: http://www.macports.org/
[bug]: http://trac.macports.org/ticket/32097
[mono]: http://www.mono-project.com/
[redux]: http://reliablybroken.com/b/2011/12/sharpziplib-and-mac-redux/

Using MacPorts behind a firewall

I failed to persuade [MySQLdb][mysqldb] to build on a [Mac OS X Server 10.5.8][1058] install using the system [Python][python] + [MySQL][mysql] installation. So I turned to [MacPorts][macports] where I know I can get [Django][django] + all the bits working without much hassle (but with much patience).

The next problem was that MacPorts couldn’t update because [rsync][rsync] was blocked by the corporate access policy. Fortunately plain HTTP is permitted outbound. Here’s how to use a local ports tree.

Install MacPorts using the disk image for 10.5.

curl -O http://distfiles.macports.org/MacPorts/MacPorts-1.8.2-10.5-Leopard.dmg
hdiutil attach MacPorts-1.8.2-10.5-Leopard.dmg
sudo installer -pkg /Volumes/MacPorts-1.8.2/MacPorts-1.8.2.pkg -target /
hdiutil detach /Volumes/MacPorts-1.8.2

If the MacPorts install directories are not in your $PATH environment, you can add them to your `.profile`. This change will not take effect until you start a new terminal session.

*(Updated to keep variables as-is as suggested by commenter Bruce).*

cat >> ~/.profile <<\EOF PATH=/opt/local/bin:/opt/local/sbin:${PATH} MANPATH=/opt/local/share/man:${MANPATH} EOF After you have installed MacPorts, create a directory for the ports tree and check it out using [Subversion][svn]. sudo mkdir -p /opt/local/var/macports/sources/svn.macports.org/trunk/dports cd /opt/local/var/macports/sources/svn.macports.org/trunk/dports sudo svn co http://svn.macports.org/repository/macports/trunk/dports/ . N.B. In the last line beginning `svn co ...` the trailing directory separator is significant! Now tell MacPorts to use the local checkout rather than rsync. Edit `/opt/local/etc/macports/sources.conf` and add a new line to the end with the path to the ports tree, then comment out the previous line that uses rsync. Here are the last lines from my configuration: #rsync://rsync.macports.org/release/ports/ [default] file:///opt/local/var/macports/sources/svn.macports.org/trunk/dports/ [default] Finally you must create an index for the tree (otherwise you will see messages saying "Warning: No index(es) found!"). cd /opt/local/var/macports/sources/svn.macports.org/trunk/dports sudo portindex Now go do great things. [mysqldb]: http://mysql-python.sourceforge.net/MySQLdb.html [macports]: http://www.macports.org/ [1058]: http://www.apple.com/server/macosx/ [mysql]: http://www.mysql.com/ [python]: http://www.python.org/ [django]: http://www.djangoproject.com/ [rsync]: http://samba.anu.edu.au/rsync/ [svn]: http://subversion.tigris.org/