GNU Classpath


Over the last week, I’ve been getting Gentoo and Free Java up and running on my new x86_64 box, a process which has culminated in the creation of my own overlay:

http://fuseyism.com/hg/libre_java_overlay

For those unfamiliar with Gentoo, an overlay is an additional set of packages (known in Gentoo as ebuilds, as for a source-based distribution the packages are essentially build scripts) which can be placed over the top of the main system tree to provide newer/better versions of existing builds and completely new ones too.

The Libre Java overlay includes a build for GCJ 4.3 (adapted from the one for an alpha snapshot in the java-gcj-overlay) and one for IcedTea6. Unfortunately, Gentoo’s Java support seems incredibly broken — the main stable and experimental (~) distributions don’t include OpenJDK or IcedTea, and the stable versions of GNU Classpath and VMs like CACAO and JamVM are ancient. GNU Classpath is still on 0.90, which is older than the one in Debian stable. It also tries to pull in the proprietary JDK by default; I recommend Free Java Gentoo users add:

dev-java/sun-jdk
dev-java/ibm-jdk
dev-java/jrockit
dev-java/diablo
dev-java/sun-jre-bin
dev-java/blackdown-jdk

to package.mask to avoid accidentally installing proprietary software on their machines. Unfortunately, stable versions of portage have yet to honour any license scheming, although it is in the unstable version. If you use Gentoo, feel free to try out the ebuilds from my overlay and give feedback. To use it, just get Mercurial (emerge mercurial), clone the repository:

hg clone http://fuseyism.com/hg/libre_java_overlay

and then add the following to /etc/make.conf:

PORTDIR_OVERLAY=<location you downloaded libre_java_overlay to>

The build process for IcedTea6 is fully documented on the IcedTea wiki.

I read Mario and Roman’s posts this morning and they inspired me to post too, after realising that there was a lot of stuff that’s been going on that I also hadn’t blogged about. Firstly, Google Summer of Code closed its doors to student applications on Monday (well Tuesday really, here in Europe) and we were rather disappointed to find only two applications for GNU Classpath, both for java.util.Scanner. This is a real pity, as I think we had some really good ideas on the list (and even more interesting ones were left to one side after AICAS didn’t get in as a mentoring organisation).

I think students picked Scanner because it’s used on many undergraduate courses these days (something of which I was blissfully unaware, as it’s years since I did any introductory Java course or read an introductory book). Unfortunately, the future of such an idea is really dubious, as I explained to our first applicant online, as there is already an implementation (no idea how good or complete) by a student of Christian Thalinger (twisti) which we hope to get into the codebase, once the legalities are sorted out. Equally, we are looking at using BrandWeg to get the OpenJDK version; this has only really ground to a halt because I found we’d need to either update our regex implementation or also bring in the OpenJDK one and I simply haven’t had time. Again, BrandWeg was on the ideas list, but no takers :(

Personally, I’ve mainly been looking at IcedTea recently, in preparation for the OpenJDK challenge work. I haven’t really blogged on the details of this yet, but my first port of call will be to take an OpenJDK/IcedTea build and attempt to get the class library from that to work with one of our Free GNU Classpath VMs. Rather than trying to build the Sun interface into that VM, I want to use that process to figure out the best way to create a more well-documented VM interface including support for VMs that don’t want to go the native route for the VM interface. I intend this to be an interactive process so I’ll be posting results and hoping for feedback as we go along. As the OpenJDK challenge infrastructure gets sorted, I expect this will take place under the auspices of an OpenJDK project. I’ll be tagging appropriate blogs with the ‘VM interface’ category so feel free to track them.

One question that does spring to mind is whether the challenge projects are intended to work with OpenJDK or OpenJDK6. IcedTea work has pretty much shifted to OpenJDK6 (in the form of IcedTea6) with good reason; distros want to ship a stable Free equivalent to JDK6, not an early alpha of JDK7 (which doesn’t yet even have a JSR attached to it). Ideally, I think we should maintain both, as the IcedTea porting process seemed to suggest the differences weren’t too major. But feedback here would be welcomed. I’d also like to make the new VM work easy for others to test, so hopefully some integration with IcedTea will be possible there too. I’m very impressed with IcedTea so far, especially Gary’s zero port (as my previous blogs hopefully illustrate). There are some niggles, but lots of eyes and people testing it in different environments will fix these. It’s great to have OpenJDK on PPC64 for one thing!

To close, I notice that people are moving around in the Free Java world (as my reference to Mario’s title in mine hopefully indicates). First, Tom Marble left Sun earlier this year, and now we find that Dalibor has successfully stepped into his (hopefully clean) shoes. To me, there doesn’t seem to be a more appropriate replacement, though it does make his governance board position interesting… Mario has also now moved to AICAS, so it seems like most people are getting new jobs and different roles in the new Free Java world brought about by OpenJDK. Things will change for me as well soon, as I’m due to finish my PhD here in October (well the funding runs out at least, which means I need some alternate source of cash at any rate). So it will be interesting to see where I am in a year from now too…

It’s that time of year again! This time, I’ll hopefully be mentoring some students instead, especially after my OpenJDK Challenge proposal was accepted. :)

If you’re a prospective student for SoC 2008 and would like to work on Free Java, then please come and join us on either:

Want to find out more? Then mail classpath@gnu.org or hop into #classpath on FreeNode IRC (same network as #gsoc this year).

It was FOSDEM this weekend, and once again the Free Java hackers (now including many Sun employees) met up in Brussels to discuss the past, the present and the future. We had an extremely good turnout, better than last year (the only other year I’ve attended so far) and with nearly all the Free Java people there (notably MIA were Tom Tromey and Michael Koch). Tom Marble did a great job of encouraging everyone to give a talk, so, unlike last year, I think we heard from everyone working in the Free Java space. It certainly left me with plenty to think about.

Just before leaving for FOSDEM, I got 0.97 out of the door (or tried to — there were problems doing the upload again, but this should be sorted now for next time). In hindsight, this was a bad idea as I was then very tired for most of the trip and, in conjunction with having a bit of a cold, this meant I didn’t enjoy the weekend as much as I’d have liked. Anyway, the main point about the release was that it was supposed to represent that GNU Classpath was still around, even if not going strong, and not merely a dead carcass for people to pick the bones of. However, FOSDEM made me realise that we may be seeing the end of GNU Classpath sooner than I thought. With the planned BrandWeg hacking session on the Friday, that was initially going to be my main focus of the weekend, but from the talks it seems most people are further along the road to OpenJDK than I thought. Maybe the BrandWeg idea is coming about six months too late and we should have gone there around the same as IcedTea. As is, CACAO, JNode and IKVM.net are already using OpenJDK, and JamVM, Kaffe and JikesRVM all have their sights on going there too. There was unfortunately no real discussion of GCJ, so it may just be that, with RedHat’s focus on IcedTea, there will be no further development there at all, which is a shame. Overall, FOSDEM gave me the impression that our Free Java community now want OpenJDK and have grown out of their ‘toy’, GNU Classpath. I’d be interested to know if others think this is true too.

The most interesting talks for me, however, were those at the end where the TCK and the JCP was discussed. As Dalibor has mentioned in the past, I couldn’t help thinking we might have targeted the wrong opponent. While it’s nice to have the more stable, mature codebase of OpenJDK out in the open, when all is said and done, it’s just another implementation (and in some ways a competitor for Classpath, at least in terms of the attention of developers). The fundamental problem is the JCP, which still largely turns out proprietary TCKs and RIs (Doug Lea’s JSR166 being the notable exception to the norm). In the long term, a truly Free Java platform is not just about implementations but about the specification and an open process surrounding that. This is perhaps what has been the barrier to people getting involved in OpenJDK; because OpenJDK is largely an amalgam of reference implementations of JSRs, all FOSS hackers can do outside these JSRs is bugfixes which, while necessary, don’t set their hearts on fire. A true Free Java community can only really work with participation in open discussions that determine its future and with a Free Software TCK. I think Andrew Haley explained the benefits of the latter very eloquently in his talk, where he demonstrated how a number of annoying bugs in our code could have been spotted much earlier and easier had the TCK been available. Can we not separate the test suite from the certification process in some way, such that the tests can be developed in the open (and can thus be more readily trusted) and used by more than just licensees? Anyone want to join the JCP and make an impact?

In more technical news, not only is Classpath 0.97 out, but I successfully compiled it on Nevada (via a SunRay connection) this afternoon… :)

Well, this year is trotting along quite nicely so far and there have already been some quite interesting developments. Plus it’ll soon be time for FOSDEM again! Probably the most major thing for me is the continued development of OpenJDK-related projects. After a discussion with Dalibor via IRC a couple of weeks ago, we launched a new project called BrandWeg. This is effectively the inverse of IcedTea; we take Classpath and try to patch its holes using OpenJDK as the plaster.

It’s notably different as a repository in that it contains no source code, only patches. Instead, the build downloads Classpath from CVS or a tarball, and then the requisite OpenJDK sources from either Mercurial or a tarball. At the moment, it just merges in the jaxws stack, as this is completely unimplemented in Classpath and separated in the OpenJDK. As such, it seems like it should be easy to add this in by merely putting the sources into external and linking it into the build system. Unfortunately, the code may be in different repositories but they are by no means orthogonal unfortunately. Trying to build with jaxws in there makes the build scream for mercy — among the errors, there are ones looking for a HTTP provider in com.sun (presumably tucked away in the main JDK sources) and some evil direct use of the com.sun.org.apache XML providers. So it’ll need a bit of patching before we have our first BrandWeg build.

I’ve also returned to a little work on IcePick, after doko has started trying to build native GCJ binaries from its output. Together, we found and patched a few bugs and I’ve finally implemented better wrappers for the tools. Notably, these don’t depend on the -Xbootclasspath option which I picked up from the Classpath tools but which doesn’t work with gcj, and they also handle the -J options. The wrapper is now a little bit of C code that is compiled with different defines depending on the required tool.

Feel free to check these and let me know what you think.

Finally, the last bit of news concerns OpenJDK again. Through the last year or so that we’ve been working with the new OpenJDK development, our main contact with Sun has been through Tom Marble. He’s proved to be a great guy to work with, always patient enough to listen to all our quibbles and our ongoing complaints and whines about the glacial progress of the OpenJDK community development. So, it’s with some sadness that we found out this week that he’ll be leaving Sun soon and heading off for pastures new. I hope he appreciates how much of an asset he has been to us GNU Classpath developers in our trips into the unknown world of the OpenJDK and I’d like to join with the rest of the Free Java community in thanking him for his tireless work and wishing him all the best in the future. Thanks Tom!

Well it’s now 2008 (unless you’re in the US of course) and traditionally a time to reflect upon the year just gone. Well, it’s certainly been one of movement and change in the GNU Classpath community, if also one of very little code hacking. Of course, the biggest event of the year was the release of the OpenJDK source code in May; about 95% of the JDK source code base was made available under the GPLv2, the rest both then and now being filled by binary blobs, which are slowly decreasing in number.

However, back in January 2007, we were still waiting for this to happen. As I recall, we’d just (gone through the generics merge, migrating the contents of the branch on to the main tree and over the Christmas period I added quite a lot of 1.6 code. Sun’s javac compiler was open-sourced however, as part of the November announcement, and we soon had support in GNU Classpath for building with it.

In February, GNU Classpath was again at FOSDEM and I finally attended after missing out on the last two years. It was good fun and notable for our collaboration both with the DevJam folks and the OpenJDK community (AKA Sun). Everything was very much still ifs and buts however, because I don’t think anyone knew what would really happen when the floodgates opened and the JDK was released. Of course, GNU Classpath will again be at FOSDEM this year, and apparently with even closer collaboration with OpenJDK — try and even spot a mention of GNU Classpath on those pages… Hopefully, I’ll be there again too.

The year only saw two Classpath releases, which is sort of representative of its ongoing decline I guess. 0.95 was released shortly after FOSDEM in April, a delayed and very necessary release which include the generics move and the other stuff I mention above. Getting a GNU Classpath release out the door has become more and more of a struggle over this year and, in a way, has become synonymous with a pregnant mother giving birth, as a team of developers stand round screaming ‘push’ until eventually the release emerges. Our release of 0.96 in October certainly felt like that to me anyway. On that note, congratulations to both Michael Koch and Roman Kennke who both (or should I say, their respective partners) had children this year.

May of course saw the release of the OpenJDK source code and the inevitable hordes jumping on and pulling it about took place with the same speed and haste we saw on the release of javac. This has eventually surfaced in the form of IcedTea from a number of GNU Classpath/RedHat folks, a build framework for the OpenJDK which, most importantly, serves it without the need for proprietary code (given that the current OpenJDK build has no real advantage over a proprietary JDK drop unless you want to tinker). This was produced by splicing in bits of GNU Classpath code. It’s a whole deal more easier to use than plain old OpenJDK but still not a walk in the park. This is especially true since Sun split up the build process and made parts of it depend on Ant

It’s quite sad to say, but OpenJDK is still not what we all hope it will be. Thankfully, live Mercurial repositories are now available as of December, but there are still too many walls, as I mentioned before. It’s like the JDK was something that popped out of a little black box, but now someone has taken the lid off the box so you can see the little worker ants running around inside and making it all happen. But there’s not yet anyway to get inside the box ourselves. New ants need to be able to join in the fun. We’ve already seen one of the goals I mentioned in August realised (the open repositories), and hopefully more will change in 2008. Sun’s innovation awards are both a realisation of this problem (common to both OpenJDK and many of its other open source projects, which still have a predominantly Sun ethos) and hopefully its saviour.

It smells a little like Google’s Summer of Code and indeed Tom Marble was asking myself and Mario about this on IRC. This year of course saw me tackling JikesRVM for GSoC, trying to implement the VM-level management extensions. A lot of this work was done, and following GSoC, I was made a member of the JikesRVM team. However, the patches still haven’t gone in to the mainline, due to a combination of the release process (some of these are likely to be destablising) and finding time myself to test the stuff on the main branch. This hasn’t been helped by a re-emerging problem with building JikesRVM using Classpath; this same problem plagued my initial month or so on the project when I discovered it seemed to rely on proprietary software to work…

So altogether it’s been an interesting year. Not only have I worked on both the old (continuing with GNU Classpath and also pushing out a release of GJDoc) and the new (launching the IcePick project to build the OpenJDK tools separately) over the last year, but I’ve also sadly seen the community enter something of a demise as people start to move on to other things, especially IcedTea and OpenJDK. What will 2008 hold? Maybe I’ll find myself working on OpenJDK too…

One of the things I’m currently working on is getting Classpath to a state where the java.lang.management stuff I’ve worked on becomes useful i.e. where we can attach to a VM either locally or remotely and get pretty graphs of things like memory usage via JConsole.

I’m tracking this via a couple of meta bugs. PR34277 represents our final goal of being able to monitor Classpath VMs and use Classpath to monitor other VMs, while PR34078 is looking at simply getting JConsole up and running on GNU Classpath. I’ve had some progress with this today. I’ve closed three bugs (two of them Swing/AWT ones which I’m quite chuffed about) and finally got some visible progress:

JConsole running on GNU Classpath

So far so good. The menus don’t work though (PR34581) and there’s also a problem once you click Connect (PR34582). The latter looks easy enough to solve (we just need to be checking for null!) and the menu thing is more of an annoyance than a blocker (the keyboard shortcuts on the menu items do work). However, I feel there is still a lot more to do before it works.

With any luck, Classpath 0.96 will be out tomorrow evening. Please get your patches in by 9am UTC tomorrow. Any testing would be much appreciated!

Now that SoC is finished, I’m looking at what to do next and a lot of things within the Free Java world are competing for my attention. Primarily, we need to get another Classpath release out the door as 0.95 was released in April. There have been quite a few bug fixes since then, and it would be nice to make these more widely available. Also, we plan on releasing a new version of JikesRVM pretty soon, and it really should ship with the latest Classpath. One issue that I believe should be fixed in CVS (e-mails this week seem to be saying something to the contrary) is buidling using OpenJDK’s javac. For JikesRVM, this would mean it could drop its dependency on downloading, building and using ecj. A plan for this is now on the Classpath wiki.

On the subject of JikesRVM, I’ve now been made a member of the core team and I’m working towards integrating the JMX stuff (still on the SoC branch) into the main tree. For the imminent release, I think this will just be the parts that don’t impact too heavily. There are some changes to the threading code involved that I think need to wait until post-release and more review/performance testing.

In the long run, I’d like to see Classpath being able to run JConsole, which means supporting the remote part of the JMX specification. This should hopefully mean a new blast of code into the Classpath repository which has been dormant of late. I also need to look further at IcePick, merging my work on building the Java tools (javac, javadoc, etc) with Dalibor’s SoC work for javac. There’s also more work that could be done on GJDoc, if we’re still maintaining that, and I’d like to look and see if there’s anything that I can do to contribute to OpenJDK yet (the modules stuff Dalibor mentioned to me looks interesting). Lots to do, I should file a bug request for a 48-hour day enhancement…

We’re just about to enter the last three hours of Google’s Summer of Code 2007. Pens down (or should that be hands off keyboard…?) is at 7pm UTC today. With respect to my work on JikesRVM, things seemed to have gone relatively well. Last night, I completed the first bash at the threading bean (some methods are implemented, a few are still stubs), so my branch now includes an implementation of all the VM interfaces. I feel the task has been pretty successful; I’d say I know my way around the code base pretty well now and I can see how the few things that are left could be implemented. Those that are left over will take longer, mainly because they involve capturing data which isn’t captured at present. This means we need to insert appropriate hooks into the code base in a careful way so that performance doesn’t degrade too much.

I think communication could have been a little better, but I’m to blame for that a lot. Unfortunately, we PhD students may be eligible for SoC but I don’t think we’re that well suited because we can’t devote 100% of our time to it. Had I had more time to give, I no doubt would have achieved more and interacted more with the community than I managed to. It’s also a very different atmosphere, and I’m not used to working on code at the (relatively) slow pace of e-mail correspondence. One of the joys of hacking on Classpath has been interacting on IRC with my fellow developers and the camaraderie that ensues there. With e-mail, there’s always a feeling that the response you received didn’t quite achieve what you want and now you have to go through the whole rigmarole again of sending and then waiting for someone to check and respond.

Another advantage of my work has been that JikesRVM can now be built on a Free stack again. That took some work, and sidetracked me quite a bit, but I think it was worth it. I still want to try getting the thing self-hosting, but I believe that should be fairly trivial. The main thing lacking at present is rvm’s support for acting like a ‘java’ clone. It lacks some options to do with modifying the bootclasspath, but I can see how they can be implemented.

The interesting bit to come will be merging my changes on to the main branch and hopefully working on there in future. I’m still not sure how that will work, or what hurdles I’ll have to jump through. I’ll guess I’ll find out in the days to come. I’ve really enjoyed working on the project and would like to continue contributing.

How OpenJDK Can Become GNU Classpath

Strange heading I’ll admit, but this is something I hinted at a little above and in previous blogs. It’s one thing to release a blob of source code as FOSS, and that takes a lot of work. Again, well done to Sun for what must have been a huge task with about 6 million lines of code. But making it into a FOSS project is a completely different ball game; just ask Apple ;)

Some rather cynical folks could say that Sun have already achieved their goal; the release of a GPLed version of Java means these freaky GNU hippies can go off and play with that somewhere and not compete anymore. However I feel this really is a positive move by Sun and that they do intend to turn Java into a true collaborative development. I do think there are a number of steps that need to be done to achieve this though:

  1. The code base needs to be writable; I know this is in the works, the sooner we have an active Mercurial repository rather than a read-only subversion one, the better.
  2. External developers need to be able to contribute to this code base through an open process which encourages public development and (fairly) rapid feedback. Classpath has recently had a very quick patch turnaround, but admittedly things were also getting a bit laissez-faire.
  3. The Sun people then need to work in the open with external contributors, so we can work together and not against each other. The Classpath folks have already done some astonishing things from getting Freetype working independently to getting the OpenJDK onto PPC, Alpha and S/390 (via CACAO). If we all worked together, things would be even better.
  4. There need to be some clear goals. You won’t get many contributors if you expect them to just rattle around a messy bug database. There need to be some clearer tasks to work on, with some low-hanging fruit to get people started. This also means having the JSRs not behind horrible licenses…
  5. Most importantly, let’s have *FUN*!

« Previous PageNext Page »