Deploying with the LWJGL Applet Loader 3 - Compression

Introduction

Basics - Part 1

Customisation - Part 2

Compression - Part 3

Polish and Tweaks - Part 4

Java applications/applets are usually distributed as jar files (compressed zip files), this has become a rather old and dated compression format. Since the AppletLoader works like a custom plugin it easily allows a custom distribution format to be used in addition to standard jars.

LZMA is currently one of the best compression formats and performs much better then standard zip. Java 5+ added support for Pack200, this lossy compression scheme specifically targets class files and uses various tricks to collapse and merge them, this allows them to be compressed much more effectively. When used together, Pack200 and LZMA allow very good compression of jar files.

The standard lwjgl.jar at maximum zip compression is about 800kb in size. Using Pack200 and LZMA on the same jar shrinks it down to about 200kb. That's a very impressive 75% reduction in size.

Creating pack200 and lzma compressed jars is bit of a pain to get this right the first time but the end results are well worth it. This tutorial will attempt to run through the steps you need to get this to work.

Jars

Before using pack200/lzma you need to create your jar file(s) as normal as this is the format they will be extracted to and stored as once they are downloaded by the AppletLoader. Remember you must use uncompressed jars otherwise LZMA compression won't be fully effective.

Signing

You must sign your jars before applying pack200/lzma compression to them. However there is a pitfall you need to watch out for. Since pack200 is a loosy compression format and not lossless, when you sign a jar and then apply pack200 to it there is no guarantee that the class structure will remain the same when you unpack it. This has the effect of causing the signature verification of the file to fail. To avoid this you need to normalize the jar by using repack on it before signing. This will ensure that once you unpack, your jar is the same as before and the certificate remains valid. The Repack tool is available with the pack200 tool and creates a normalized jar.

Pack200

To create pack files from you can use the JDK's built in tools to apply pack200 to jars files. To simplyfy things further you can use tools like Scar (build system) or the Pack200 Ant Task.

Pack200 is only useful for jars with class files inside them, so no need to use it on the jars containing the native files as it won't have any effect.

Since pack200 is built into all Java 5+, you won't need any extra libraries to unpack pack200 files.

LZMA

LZMA is occasionally confused with the 7z format but do note that its different as lzma on its own can only compress a single file at a time while 7z is a container format for lzma.

Once you have your jar or jar.pack file(s) you simply use your tool of choice on them to create jar.lzma or jar.pack.lzma files.

To use your lzma file(s) with the appletloader you need to include the lzma decoder (lzma.jar) with the AppletLoader, this will allow it extract lzma files. lzma.jar (comes with the LWJGL download bundle), you simply add this to the codebase parameter in the applet tag in addition to lwjgl_util_applet.jar.   ...

To create LZMA files there are a number of tools you can use. Such as the Jlzma.jar (a command line tool), Scar (build system), Lzma SDK or the Pack200 Ant Task.

Automation using a build system

You will notice that there are a lots of steps you need to go though to get it all working. Namely the following:

1) Create uncompressed jars 2) Repack jars 3) Sign jars 4) Pack200 jars 5) finally apply LZMA to the file

This can becomes pretty unmanageable after a few goes at it. You will definitely want to use a build tool to make this easier and quicker. Some build tools that you can use include ANT, Maven or Scar.

Removing the Unused Bits

Compression can only save you so much but another way to shrink the size of your application is to remove the bits of code and other resources that aren't used. For example the LWJGL library jar is just over 700kb and contains code covering the whole of OpenGL, OpenAL, OpenCL and various other bits. Its almost certain that your application will not be using all the code in there. You can manually delete bits you don't need but this is time consuming and tedious. However you can use tools like ProGuard to automate this for you although requires some tweaking to get right. ProGuard can also obfuscate java code thus making it even small. Typically this can shrink the LWJGL library well below 200kb (including natives) but of course depends on your application.

Further Saving by Splitting Natives by Architecture

LWJGL provides natives for both 64bit and 32bit Systems. Linux and Windows have seperate natives for both architectures.

You can package the 32bit and 64bit natives into seperate jars, so the AppletLoader only downloads the natives needed for the platform and architecture.

To do this you use the al_window32, al_window64, al_linux32 and al_linux64 parameters. These parameters will take priority over al_windows and al_linux as they will be used if provided.

  

 



Since the LWJGL MacOS natives are fat binaries, meaning PPC, 32bit and 64bit architecture natives are lumped together into the same binary it not possible currently to split the natives into separate jars. However the AppletLoader does support the al_mac32, al_mac64 and al_macppc parameters in case you have other natives which are platform specific.

Introduction

Basics - Part 1

Customisation - Part 2

Compression - Part 3

Polish and Tweaks - Part 4

Tutorial Credit - Ninja Cave