Gentoo


Lillian ran into a few minor problems trying to build ‘raw’ OpenJDK6 (i.e. the tarball direct from Sun without any of the numerous IcedTea patches, fixes and extensions) yesterday, and so I decided to give it a go as well. Unfortunately, it seems it still isn’t possible to build it out of the box on a modern GNU/Linux system. In the end, I had to apply three patches (effectively two as one was split into two by the HotSpot build changes in IcedTea). patches/hotspot/original/icedtea-gcc-4.3.patch and patches/icedtea-gcc-4.3.patch from IcedTea6 is needed to fix some issues when building with GCC 4.3 which is the default on most current distributions. In released versions, this is a single patch, but in current Mercurial and the upcoming 1.4 release, the HotSpot changes in all patches are split off to enable the version of HotSpot used by the build to be changed. patches/icedtea-no-bcopy.patch was needed to remove some local defines of some BSD functions such as bcopy. There was some noise about taking this upstream on the mailing lists, but it’s not yet in a tarball it seems.

With these patches, I could build as follows:

$ mkdir openjdk
$ tar xzf openjdk-6-src-b14-25_nov_2008.tar.gz -C openjdk
$ cd openjdk
$ patch -Np1 < $ICEDTEA_SOURCES/patches/icedtea-gcc-4.3.patch
$ patch -Np1 < $ICEDTEA_SOURCES/patches/hotspot/original/icedtea-gcc-4.3.patch
$ patch -Np1 < $ICEDTEA_SOURCES/patches/icedtea-no-bcopy.patch
$ cd control/build
$ unset JAVA_HOME
$ unset LD_LIBRARY_PATH
$ JAVAC= LANG=C make ALT_BOOTDIR=$CURRENT_ICEDTEA_INSTALL \
 IMPORT_BINARY_PLUGS=false ANT=/usr/bin/ant ANT_HOME=/usr/share/ant

...sometime later...

>>>Finished making images @ Fri Jan 30 10:59:43 GMT 2009 ...
make[1]: Leaving directory `/tmp/openjdk/jdk/make'
Control build finished: 09-01-30 10:59
$ ../build/linux-amd64/j2sdk-image/bin/java -version
openjdk version "1.6.0-internal"
OpenJDK Runtime Environment (build 1.6.0-internal-andrew_30_jan_2009_10_46-b00)
OpenJDK 64-Bit Server VM (build 11.0-b17, mixed mode)

You can build a bit quicker if you make use of parallelisation which I neglected to do here; add

ALT_PARALLEL_COMPILE_JOBS=$PARALLEL_JOBS  HOTSPOT_BUILD_JOBS=$PARALLEL_JOBS

where PARALLEL_JOBS is how many processes you want to run simultaneously. The usual rule is number of cores plus one. IcedTea of course supports doing this too; just add --with-parallel-jobs=$PARALLEL_JOBS when you run configure and it will add the necessary make wizardry for you.

For the configuration, CURRENT_ICEDTEA_INSTALL should point to your existing IcedTea install. This is the only way to build OpenJDK6; I didn't try it this time round, but I know from experience that it still needs a fair few patches to remove Sunisms from the OpenJDK source code in order to allow a GNU Classpath JDK like GCJ to be used instead. This shouldn't be a problem though; you'll find IcedTea in Fedora (yum install java-1.6.0-openjdk), Gentoo (emerge -v icedtea6-bin), Ubuntu and Debian testing (aptitude install openjdk-6-jdk). The magic path is usually something like /usr/lib/jvm/blah where blah is java-1.6.0-openjdk on Fedora and /usr/lib/jvm/icedtea6 on Gentoo.

ICEDTEA_SOURCES points to a copy of the IcedTea tree. If you get this from a release tarball rather than hg, then you don't need to try finding the second gcc patch... you'd have a hard time doing so ;)

The OpenJDK build doesn't like environment variables like LD_LIBRARY_PATH and JAVA_HOME being set. It also complains about you having any fancy modern locale set in LANG so it's simplest just to run make with LANG=C. The JAVAC environment variable is a funny one; it allows the path to javac to be overridden for the build, but it doesn't only override the binary path as it should, but also drops all the necessary memory and classpath options passed to it. Thus, I don't see how any build which sets JAVAC could work... Gentoo seems to like to set both JAVA_HOME and JAVAC for some reason, so make sure they are unset before you build.

Of course, with the resulting build, you'll be missing a lot of features that IcedTea provides...

  • A web plugin
  • Java web start
  • PulseAudio sound support
  • Support for architectures like PPC, ARM and MIPS via Zero or CACAO
  • A faster more recent HotSpot if you use the current Mercurial tree or the upcoming 1.4 release
  • Lots of other lovely stuff provided by our patches including a more up-to-date version of Gervill, the results of the XRender pipeline project and support for testing your build with JTReg.

You also avoid the pain of having to remember those crazy make variables; it's just ./configure; make on most modern GNU/Linux systems; file a bug if that's not the case.

I still feel that the IcedTea project does a very important job in taking the raw materials provided by Sun and turning them into something useful. That's why you'll find it's IcedTea being shipped with all those distros, and not 'straight' OpenJDK (which in most cases actually means the version that may-eventually-be-1.7). Thanks to all the great developers for their continued efforts and for the support of Red Hat on this effort.

The other thing about your own build is you can't of course actually call it 'Java'; one of the things you need to do before you can is run it through the Java Compatibility Kit. It has lots of tests your build has to pass. Unfortunately, Sun still keep this as a horrible proprietary blob of code which means you have to sign up to get a copy, be 'approved' and then work on your testing in a dark clammy room in secret. Fortunately, the kind people at Red Hat have already undertaken this momentous task for you, so you can just grab one of the OpenJDK builds in Fedora which has already passed. Thus, according to Sun's FAQ, the Fedora binaries 'are compatible with the Java(TM) SE 6 platform' :)

Let's hope things continue to improve in the direction of making more things in the Java world open and Free. In the Free Java room at this year's FOSDEM (happening just next weekend), you'll be able to find out all about what's coming next and meet some of the brave people forging the new frontiers...

Happy hacking! :)

Congratulations to the IcedTea team on the release of 1.3 (finally)! This release sees a number of significant new features, notably the availability of the cacao and shark builds, which will hopefully give more usable VMs on architectures other than x86, x86_64 and SPARC.

Gentoo users can now obtain a 1.3 ebuild from java-overlay:

$ layman -a java-overlay
$ emerge -v icedtea6

Please report bugs to the Gentoo bug database.

Congratulations to the IcedTea team (Lillian Angel, Gary Benson, Tom Fitzsimmons, Joshua Sumali, Andrew Haley, Mark Wielaard) on another release. The OpenJDK version is only one drop further on (b09 as opposed to b08) but there are lots of other IcedTea related changes including the import of MIDI support via Gervill and a Linux/SPARC HotSpot port, not to mention numerous fixes and jtreg-based testing. Gentoo users can already find an ebuild for this in my overlay: http://fuseyism.com/hg/libre_java_overlay.

Unfortunately, creating this ebuild immediately showed up a few issues with the release. The first was rather peculiar, it seems the jar executable from IcedTea 6 1.1 couldn’t handle the @ option used to build the rt.jar file:

(cd /var/tmp/portage/dev-java/icedtea6-1.2/work/icedtea6-1.2/openjdk-ecj/control/build/linux-amd64/classes
&& /bin/cat /var/tmp/portage/dev-java/icedtea6-1.2/work/icedtea6-1.2/openjdk-ecj/control/build/linux-amd64/tmp/jarfilelists/rt_jar_list
| /usr/lib/jvm/icedtea6-1.1/bin/jar c0mf@
/var/tmp/portage/dev-java/icedtea6-1.2/work/icedtea6-1.2/openjdk-ecj/control/build/linux-amd64/tmp/manifest.tmp
/var/tmp/portage/dev-java/icedtea6-1.2/work/icedtea6-1.2/openjdk-ecj/control/build/linux-amd64/tmp/rt-orig.jar
           -J-Xmx896m -J-Xms128m -J-XX:PermSize=32m
-J-XX:MaxPermSize=160m)
Illegal option: @
Usage: jar {ctxui}[vfm0Me] [jar-file] [manifest-file] [entry-point]
[-C dir] files ...

Interestingly, using gjar does work, so for the time being I’ve enforced the use of gcj-4.3 in the ebuild, and used gjar from this. Patching it so that it becomes /usr/lib/jvm/icedtea6-1.1/bin/jar c0mf should also work.

The other issue I ran into was with the plugin option. Gentoo’s econf system specifies all the configure options it uses even if the values are the defaults, due to the way it is automated. So even though the plugin is enabled by default, –enable-gcjwebplugin still gets passed to the build. However, it seems this actually disables it for IcedTea6. I patched this as:

diff -r 3fe8a0881e86 configure.ac
--- a/configure.ac      Wed May 28 11:29:51 2008 -0400
+++ b/configure.ac      Fri May 30 00:53:44 2008 +0100
@@ -101,13 +101,13 @@ AC_ARG_ENABLE([gcjwebplugin],
 AC_ARG_ENABLE([gcjwebplugin],
              [AS_HELP_STRING([--disable-gcjwebplugin],
                              [Disable compilation of browser plugin])],
-             [ENABLE_PLUGIN="$val"], [ENABLE_PLUGIN='yes'])
+             [ENABLE_PLUGIN="${enableval}"], [ENABLE_PLUGIN='yes'])
 AC_SUBST(ENABLE_PLUGIN)

 AC_ARG_ENABLE([docs],
              [AS_HELP_STRING([--disable-docs],
                              [Disable generation of documentation])],
-             [ENABLE_DOCS="$val"], [ENABLE_DOCS='yes'])
+             [ENABLE_DOCS="${enableval}"], [ENABLE_DOCS='yes'])
 AM_CONDITIONAL([ENABLE_DOCS], [test x$ENABLE_DOCS = xyes])

 AC_ARG_WITH([icedtea],

This wouldn’t have been noticed because usually you either stick with the default (which does turn on the plugin) or, if an option is specified, you disable it (which happens). The problem is that specifying either form of the gcjwebplugin option disables it, including the one that’s meant to enable it.

With these two issues fixed, IcedTea6 1.2 can be installed on Gentoo.

It seems a lot of projects and distributions are seeing new releases either now or in the very near future. This week, we had a very quiet minor release of GJDoc, the GNU Classpath equivalent to javadoc. 0.7.9 includes a few changes that were previously only available in CVS, but the main one is a small fix that allows Classpath 0.97.1 documentation to be built. Our minor .1 release for 0.97 fixed a bug where the JSR166 code was not being included in the documentation build. With this fixed, it turns out gjdoc would no longer build the documentation as java.util.concurrent.TimeUnit is a rather complicated enumeration that our hacks can’t bypass. Michael Koch, in packaging GJDoc for Debian, was kind enough to point out that having the current release of GJDoc not being able to build documentation for the current release of Classpath was a bad thing. A quick release fixed this by pushing out the fix I made for this issue back in March. Of course, you can now use javadoc for IcedTea/OpenJDK to build the documentation instead; with another Free JDK about, there’s no need to just rely on GJDoc.

I do wonder what the long term future for GJDoc should be. It only works with GNU Classpath at present through a nasty bunch of hacks which cause the parser to skip chunks of the input. It really needs a major cleanup and to be made to work properly with 1.5 code. Thomas Fitzsimmons suggested we should merge it into the GNU Classpath codebase which seems a good idea, as it means we don’t run into this same revision hole we just did. However, it is worth maintaining GJDoc at all? For me, the main features it has over the OpenJDK javadoc are in speed and the look of the output. A key feature is also that it it plays nicer with Free Software i.e. it includes an option to include the source code with syntax highlighting. You can see the output for Classpath 0.97 online.

JikesRVM is also stepping up for a new release, 2.9.3, and this will be the first to showcase the new Classpath support for a non-copying unsynchronised StringBuilder. This is designed for local method usage where the builder will be converted to an immutable String object rather than leaving the method. As a result, I’ve been rushing to get it in a releasable state, as I know there’s a nasty bug lurking in the older patches JikesRVM has been using recently. I managed to do this today after we fixed a build issue. It seems the javah in OpenJDK6 outputs differently named header files to those JikesRVM implicitly depends on. We fixed this by making this dependency explicit as it should be, but perhaps this also uncovered an OpenJDK6 bug. I’m not sure where we should be filing these yet, so I just posted to jdk6-dev.

It’s also nice to hear that Ubuntu has just shipped with IcedTea6 included. Fedora 9 will also ship early next month (May 13th) with similar support and an OpenSUSE build is in the works. It’s nice to see Java support making it into the mainstream, thanks to Sun’s recent moves to make their JDK Free Software. On the less positive side, it seems that Gentoo won’t see support for IcedTea6 anytime soon. The Java Gentoo developers seem to be on a strange mission to support only the proprietary Java solutions (pretty much an inverse of what Fedora, Ubuntu and Debian do). In porting my IcedTea6 ebuild from the Libre Java overlay to their own overlays, they seem to have decided to drop support for GCJ… I’m not even going to go into how dumb this action is, as I could be here a while. Suffice to say, I don’t see how IcedTea6 can be bootstrapped without GCJ, let alone how they expect to then build it on architectures like PPC, PPC64 and ARM, as we’ve seen happen on the OpenJDK distro mailing list. It seems a very odd move for a distribution supposedly built on compiling things from source…

Over the last week, I’ve been getting Gentoo and Free Java up and running on my new x86_64 box, a process which has culminated in the creation of my own overlay:

http://fuseyism.com/hg/libre_java_overlay

For those unfamiliar with Gentoo, an overlay is an additional set of packages (known in Gentoo as ebuilds, as for a source-based distribution the packages are essentially build scripts) which can be placed over the top of the main system tree to provide newer/better versions of existing builds and completely new ones too.

The Libre Java overlay includes a build for GCJ 4.3 (adapted from the one for an alpha snapshot in the java-gcj-overlay) and one for IcedTea6. Unfortunately, Gentoo’s Java support seems incredibly broken — the main stable and experimental (~) distributions don’t include OpenJDK or IcedTea, and the stable versions of GNU Classpath and VMs like CACAO and JamVM are ancient. GNU Classpath is still on 0.90, which is older than the one in Debian stable. It also tries to pull in the proprietary JDK by default; I recommend Free Java Gentoo users add:

dev-java/sun-jdk
dev-java/ibm-jdk
dev-java/jrockit
dev-java/diablo
dev-java/sun-jre-bin
dev-java/blackdown-jdk

to package.mask to avoid accidentally installing proprietary software on their machines. Unfortunately, stable versions of portage have yet to honour any license scheming, although it is in the unstable version. If you use Gentoo, feel free to try out the ebuilds from my overlay and give feedback. To use it, just get Mercurial (emerge mercurial), clone the repository:

hg clone http://fuseyism.com/hg/libre_java_overlay

and then add the following to /etc/make.conf:

PORTDIR_OVERLAY=<location you downloaded libre_java_overlay to>

The build process for IcedTea6 is fully documented on the IcedTea wiki.