torstai 28. tammikuuta 2010

Round Pegs Through a Square Hole

Hardware integration is moving rapidly. You know, spinning around, hopping about, and generally just.. moving... rapidly.

One of our most recent hiccups has been integrating our wonderful application suite with the Window Manager. Now, don't get me wrong, we have tried to avoid an unhealthy relationship with the whole contraption, but we are limited by it wherever it has not been set up according to our liking.

One of the things we like to do is moving application windows outside the screen area, i.e. move them over the edge of the screen. This is kind of like "pushing the boundaries", or "expanding our horizons", but really its really more like ignoring the boundaries altogether...

Now you might think it strange that we should do something like that. You may say: "how are applications useful when they are located outside the screen?" The answer is, that they are brought into view again when needed. And how? Well, using a very sophisticated system, of course. And sophisticated systems R us!

This brings us to the actual topic of this entry. How to manage application windows on the screen.

As mentioned before, the system relies on XML/TCP/IP connections where IPC is required, so we already have a method for having applications communicate with each other. Secondly, we have an application development platform (Qt), which allows widgets to be moved and hidden and whatnot.

Lets begin by listing the goals that we have for window management:
- moving windows
- animating (interpolating) the locations of application windows on the screen (which is really just an application of the above)
- bringing application windows to the foreground
- hiding application windows

Now, most platforms allow application processes to handle these for themselves: they may move their windows all around the screen, bring their windows to view and hide them again. Of course, with limitations. Additionally, there is almost always a mechanism for actually "managing" all the windows on a screen in a more authoritarian manner. This is often called the Window Manager.

What we have done in our implementation, is we've used a traditional Linux Window Manager. But this doesn't have to be, and indeed will not be in some of the ported versions of this software. Instead, the moving around of windows will be performed by
the individual applications, and overseen by a process that has - for the sake of safety - the ability to kill other application processes if they start misbehaving.

All this can actually be done without touching a window manager at all! The applications are all built in a framework, which allocates windows in the first place, and thus there is great control over the goings on of application windows from there. This can, then, be used via IPC from the "Central", which is and has always been, the core of our system.

Now the mind blowing part of all this is that it allows for unparalleled portability: what system can actually boast of a full windowing system that does not require native components to perform that task?

Of course, to be fair, the applications do use native routines implicitly through the Qt libraries, but that is somebody else's headache :-)

To conclude, I wanted to share one particular example of an application window residing and appearing from outside the device's screen: the virtual keyboard.



In this example, the "TextInput" application makes a kind request for the virtual keyboard to appear. This is routed through the "Central" component, which in turn tells the virtual keyboard it should appear.

Currently the implementation is done using the Window Manager (authoritarian way), but could just as well be implemented using Qt's QWidget::move() function.

keskiviikko 13. tammikuuta 2010

GUI Architecture Revisited

The GUI architecture is designed to allow several separate applications, running in their own processes obviously, to function together as a cohesive UX system.

Secondly, the applications are expected to use platform services through a single point of access and API.

To achieve this, a TCP/IP communications method is used. Control messages (related to the GUI), service discovery, and service usage is all done through the same channel. To the application programmer, this is not revealed, but instead a collection of APIs are provided. The code for XML messaging over TCP/IP is contained within the code of each service.

So, an application will tell the GsmService client instance to dial a number, the GsmService's client-side code will send an XML-message (via TCP/IP, through "Central") to the actual GsmService process. The GsmService process then uses the underlying platform's native services (via D-Bus in the case of moblin, through CTelephony on Symbian).



Graphically, it is traditionally difficult for separate applications to interact in a seamless way. In Windows Vista, this is achieved by having everything work through HW acceleration. This allows for windows to be rotated and skewed, with their contents still updating.

In a mobile software stack, especially a portable one, the problem of drawing several elements on top of each other in a seamless way, is a more difficult one to solve. In our architecture, it is done by sending offscreen pixmaps via TCP/IP to the HomeScreen. Those offscreens can then be rotated, skewed, overlayed with something else or whatever. Seems to work very nicely!

Below are a couple of shots with separate apps running side-by-side. In the left one, a dialer application sits on top of the HomeScreen application, which draws the card boundaries. On the right, an application is partly covered by the virtual keyboard application.




tiistai 12. tammikuuta 2010

First look at the GUI Applications!

Ok, the day has arrived. Last year I promised some "swashbuckling" UI-imagery, but was prevented from uploading any concept photos then. But your wait is over now!

This is the first screenshot, but I'll post more in coming days - I promise :-)
The below image depicts the "main view", with new tweets shown as little "cards"... The contents of the tweets are not mine/ours...

The image is pretty much self-explanatory, except for the neat background, which calls for a special mention: the background represents the current time of day (color brightness) and weather (rain drops) dynamically. It can show clear weather, snow, or rain.



Little by little, we are getting closer to having a complete system running. All of this is, of course, in preparation for an upcoming mobile congress..