taking shape

My original thought on this blog was that I spend more time updating docs than writing blog posts. So it would be easier to discuss open source fun while linking to docs.

Today I am going through the uDig release process; with the goal of a 1.2.1 release.

This release should be a really good time technically:
- GeoTools 2.7-M4 (released a couple weeks ago for GeoServer)
- Eclipse 3.6.1 serving as the foundation

If you would like to help out with testing a preview of uDig 1.2.1 is available here:
- http://udig.refractions.net/download/unstable/

And our notes during testing:
- http://udig.refractions.net/confluence/display/UDIG/1.2.1

The other exciting thing about this release is a steady stream of new user visibile features; brought on by the generosity of the udig community.

This was helped along by our move to gitorious.org (and away from svn).  The move from svn was timed perfectly; Refractions was recently unable to restore our old svn repository. We were able to rescue the uDig 1.1.x codebase and it can now also be found on gitorious. 

As for the original blog idea (and a sneak peak at 1.2.1) here is documentation harmed in the making of this post:

- http://udig.refractions.net/confluence/display/EN/Raster+Style+Pages

When they came for me

Sometimes prose is not enough.

When the apple came for OS 9.

I remained silent;

I was not a OS 9 user.

When they locked up Power PC

I remained silent;

I did not own a Power PC.

When they came for Carbon

I did not speak out;

I was not a Carbon developer

When they came for Flash,

I remained silent;

I was not a Flash developer.

When they came for me,

there was no one left to speak out.

So while I am upset about apple deprecating Java; the original poem by Martin Niemöller, is about something that actually matters.

OSGeoLive Lightening Overview

I am trying out slideshare as an easy way to share the OSGeoLive Lightening Overview; I will update this post when I find a website to actually store these slides on (any idea if FOSS4G 2010 website will host them?)

OSGeo Live Lightening Overview

View more presentations from jgarnett.

I would like to thank everyone who came to FOSS4G this year; especially those who sponsored the event or setup a booth. We spend a lot of time thanking the various development communities, user communities, education community, and conference organisers - all with good reason. A big thank you to the event and project sponsors for facilitating all that we do.

foss4g 4 codesprint

updates:

• live dvd translation directories getting organised (yah!)
• udig moved to git
• udig moved to geotools 2.7 (on some kind of git shadow thing)
• geotools data store tutorial re-started as a CSV tutorial using Content DataStore

foss4g 3.2

Whew; first presentation of the afternoon WPS session was a tough one. Presentation compared the wps implementation (similar to my WPS Testing) using several clients:

• deegree xml client, 52 deegree xml client
• 52North plugin (for JUMP and uDig)
• 52North Open Layers app
• XMLSpy (for xml compliance)

And the results were not pretty; really hoping we can have a WPS shootout next year.

I really liked seeing the number of WPS clients listed in the presentation; very exciting.

I found the various implementations very helpful when I signed up to the mailing lists; they provided test servers and were good and answering my xml questions etc...

Second presentation is from deegree.

foss4g 3

Going to get out of order a bit. WPS track this morning was great! And I was so sad to miss the WPS BOF yesterday.

52N was amazing with a open layers + graphical workflow layout browser app (that defines a new process that can be executed). Both GeoServer and 52N had running demos (which is always brave); ZooWPS went through the structure of their project and seem to be taking on more languages in the future.

The big news for me was the creation of a vendor neutral public email list via the OGC for WPS instances to collaborate. One of the results of the OGC WPS test bed I participated in this year was the lack sanity for a client application trying to connect and communicate with each.  Indeed you are reduced to GeoJasn because GML requires 2 execution paths (one for schema and one for the data conforming to the schema).

I hope to set up a WPS dinner this evening; thus far GeoServer and 52N are represented; I need to hunt down Zoo project and others...

FOSS4G 1

Barcelona is off to a great start with the workshops wrapping up today (there was even a bus to gather up wayward developers; there heads stuffed full of new ideas; and herd them to the conference venue proper).

Thanks to Oscar Fonts (and the many volunteers on hand) for a smooth workshop experience.

I started off with the Geospatial for Java workshop, it was an honour to be the only traditional programming workshop in the program. Nice to keep some programming in the mix since this is an open source conference. The workshop went smoothly, in part thanks to the on site support of OpenGeo staff Justin DeOlivera and Andrea Aimie.

This afternoon I am looking forward to the OSGeo Live Lightening Overview - it was great to learn about the many new projects available on this years DVD. With a total of 44 project there is a diverse range represented including a new category of software: Disaster Relief.

In a couple more minuets the opening session will start; I am writing this in the OSGeo booth watching people flock to 52 North (they updated to a new version of GeoTools finally) and OpenGeo (they have t-shirts by product).

GeoSpatial for Java at FOSS4G

There was a great round of "I am speaking" at FOSS4G posts a couple months back. They even have a cute badge for your blog.

This year I am not exactly speaking at FOSS4G, instead I am really happy to be hosting a workshop: W-13:  Geospatial for Java

I am not exactly big on promotion; however if you have not yet signed up to workshops I urge you to do so quickly. The Geospatial for Java workshop promises to be great fun. Although it is billed as an introduction the course material is staggered allowing anyone with a bit of experience under their belt to take on an additional challenge or ideas.

I would like to thank Oscar Fonts for rounding up the workshops, hardware and instructors. Barcelona is shaping up to be an amazing experience.

Start your WPS Services

Now that I have made contact with the different groups a couple things are clear:

• Everyone is very enthusiastic about having additional clients to talk to (this is the traditional chicken and egg problem with standing up either a client or a service - you need a friend in order to have a conversation - or a chicken I guess)
• Demo services are available for anything not under active development (this is great news for me)
• While deegree 3 WPS is still technically under development they are producing a downloadable war making their service easy to test
• GeoServer documentation needs some work (sad news for me but I can fix it)
• PyWPS was recommended

Today's task is to connect to each service and make sure I can parse the capabilities document; and if things go well the describe process documents.

52 North
52 North have been very supportive with both a stable service to test against, and in standing up a service from their development branch. Their development branch makes use of GeoTools 2.6 and I am keen to hear how their transition went. Recently we have made some usability improvements for GeoTools 2.7 which will make those updating older applications even easier.

• http://giv-wps.uni-muenster.de:8080/wps/WebProcessingService

Thanks to Bastian for setting up the a server using the development branch.

ZooWPS
The ZooWPS mailing list got back to me today and quickly pointed me to both examples and a sample WPS service I can test against.  The examples confused me a bit (as the skip straight to the execute requests and rush over the whole capabilities and describe process steps).

• http://shilpa.media.osaka-cu.ac.jp/zoo/?Service=WPS&Request=GetCapabilities&Version=1.0.0&Language=en-CA

Thanks to Nicolas for the sample server to test against.

While the examples are confusing (and show a danger of just using links for data) they have a simple great picture explaining how their WPS functions.

deegree
Deegree makes a number of demonstration services available for testing; but they make use of an older 0.4 version of the WPS specification (and my dedication to standards compatibility has a limit). The new deegree 3 is implementing WPS 1.0 and has a war available that fits my needs.

• http://wiki.deegree.org/deegreeWiki/deegree3/WebProcessingService

(can I capitalise deegree if it is used to start a sentence ... or is it like "iPod" and the shape of the word matters more then silly english sentence conventions?)

GeoServer WPS Community Module

While I can quickly produce a war of the community module; my preferred method of testing is to use a very lightweight application server called "Jetty". Indeed use of Jetty is rolled into the maven build system:

cd web/app
mvn jetty:run -Pwps

The only difficulty is that the build tool maven has grown a bit responsible since I last used it and will no longer install plugins such as jetty without me modifying a couple of configuration files first. I am going to sort out what is needed and update the GeoServer docs later today.

PyWPS
A recommendation from yesterday (thanks Tom) which appears to be a contemporary of deegree in terms of years of experience. One thing that really attracts my eye when looking at a new project is:

• recent news (showing that the project is alive)
• documentation (even better if it is called course material)

And guess what is on the PyWPS home page?

2010-05-05 New course material added

New course material added to PyWPS source. See documentation for details.

I will sign up to the email list and try and introduce myself shortly.


uDig (ie client)
The other thing I am working on is user interface ideas to present the idea of external processing. My thinking thus far is to cheat - and represent external results (and if processing is still going use a progress bar as a placeholder).

If I consider it as a list of results it becomes a more interesting and productive user interface concept:

• results can be "tagged" to define ad-hoc grouping according to server, process, processing status
• results that were produced externally (such as to an ftp site) can be listed, and downloaded if needed
• using a wps could be considered "adding" a result to the list and handled using a wizard (although a wizard is not the best for interacting with the map - such as selecting a calculation area)
• I should be able to record the steps that were used to produce each result and "rerun" if needed

Web Process Service Round Up

I have a fun bit of work lined up - updating the web processing service client code in uDig.

It is no secret that I am a huge fan on the idea of Web Processing Service - I am excited about the possibilities in using a WPS as a front to a grid of computers (a strategy 52North seems to be pursuing), the ability to bundle up processes written in a number of languages (something ZooWPS is really going after).

The part I am really keen on does not seem to be tackled yet: I am very interested in chaining processes using standard diagrams such as BPEL - this represents a really nice olive branch between GIS and the business analysts that would love to know what the department is doing). There is some confusion in this area as the diagrams end up looking similar to those provided by BI tools (since GIS is used for decision making) or similar to ETL tools (since chains of processing are required).

Today am making contact with the different web processing service implementations and warning them what I am up to and generally finding out where they live and what is a good contact point for communications.

Thus far:

• 52North - 30 mins to respond to email, seems to be very active and able to link to an example WPS service right out of the gate. This is the established open source WPS solution and I am looking forward to seeing how it handles feature collections and raster processing.
• ZooWPS - no response to email yet, but the IRC channel was well populated (turns out half the members were my LISAsoft co-workers from different offices around Australia). This is the new kid on the block in the WPS space
• GeoServer WPS Community Module - no email since I had already been following that email list. The GeoServer WPS community module has been very quiet in its development but has made recent progress in the two areas I am interested in testing.
• deegree 3 is working on their second generation WPS implementation and is under active development - I may end up building from source in order to have something to test. It is great to see the continued support of WPS here (deegree 2 worked against an earlier version of the specification).

The two areas I am targeting each have their own special risks.

Features should be the bread and butter of GIS processing and we are held back in this area by the generally hap hazard support for GML. I can see nailing everything to the wall using GML and XML Schema - this is really what should be done - (since it is a data interchange format) when shuttling data between services. GML allows us to communicate the range and limits of the data and be able to negotiate differences between data models. I could see using this approach in an ETL context or when doing scientific work.

The expectations of the current crop of implementations are in a slightly different direction: focus on geometry (hey it is spatial!) and have the attributes carried along for the ride. The ZooWPS implementation also supports GeoJason which is very good for this style of ad-hoc collaboration. Even for this ad-hoc style we will need to indicate "which" geometry in a feature needs to be acted on ... so it should be fun seeing what the different implementations have provided.

Raster data is also interesting/scary. There is an answer in place for the obvious question of data size (the WPS specification accounts for this by allowing long running processes making use of FTP sites for staging results). The other question is the same one encountered by web coverage service; what does the data mean? Which bands mean what and how is your DEM height measured etc. I am really not sure if WPS is up to capturing this information; will the file format headers capture this in enough detail; or will each process need to be supplied hints to sort out how to interact with the information.

Unbuntu 10.04 VmWare Fusion missing keyboard

So this issue has been reported in a couple of places, Unbuntu 10.04 is not playing nicely with VmWare Fusion. As I understand it the keyboard is set to something that does not exist - making it very hard to login to.

Once you do manage to login everything works as normal; and you can configure the correct keyboard using:

• sudo dpkg-reconfigure console-setup

How you login to begin with.

1. Click on your user name
2. Click on the accessibility symbol; it brings up a dialog from which you can choose a visual keyboard
3. The keyboard flashes on the screen ... and disappears!

Here is the trick; it will work the next time you reboot! Restart your VM; this time when you turn on the visual keyboard it will stay up - and you can login; from there a visit to the shell will allow you to "console-setup" and life will go on.

So far Unbuntu 10.04 is very purple and the windows work well along side the mac.

Happy New Year

Happy new year everyone; and thanks for making the last year so incredible. From the supportive uDig community; to the incredible range of new features added to GeoServer; to the tutorials for GeoTools it has been an amazing year all around.

Thank you again to all of those who visited the marvellous city of Sydney for FOSS4G.