Qt 5 Under Digia – A New Chapter Beckons

You need to read the Qt blog before mine, Digia’s acquisition post is assumed knowledge for this post. I for one welcome Qt’s new corporate overlords and hope this is a lovely trolltech-style period of fantastic development; before another huge company buys Qt for millions. Qt deserves another chance at world domination in a few years.

I personally hope to continue working with the Qt project to develop QML after leaving Nokia. I’m not joining Digia, the Brisbane office closes end of August and is not part of the transaction, but with open governance I am very optimistic about trying life on the community side of Qt Development. As you may guess, today’s news also hasn’t dampened my enthusiasm about particles. Someone has to keep particles running in order to ensure Qt is the most sparkly UI toolkit around.

Some amusingly worded legalese means this could be my last chance to say bad things about Nokia 😛 . Unfortunately I don’t have anything particularly negative to say about the company. The closing of the Brisbane office has so far been a model operation, and I agree with the decision that Qt is no longer a good fit for Nokia (honestly it was always a weird pairing but I think everyone could benefit from Qt 😉 ). Nokia’s biggest failing while they had Qt was their constant strategic shifts – I watched as Qt became the messiah for at least three platforms that were publicly executed about a year later. That’s far too soon for Qt to make a difference, for those thinking we killed them. The current Nokia strategy may not be going well, but it’s still better than killing yet another platform they made a public commitment to. I could just be bitter because on the first cycle I was excited enough to encourage a friend to get an N900. That was on a deal with a two year contract, and mere months before Maemo was publicly axed and development dried up. I had several opportunities to make the same mistake again, but I had learned my lesson. I think Nokia has finally learned that lesson too. As I leave the company, I wish Nokia the best of luck with its Windows Phone 7 strategy.

This shift in Qt development focus is really gratifying to the arm-chair general in me. Now that Qt is under new management, they’re moving in the direction I’ve been most hopeful for: cross-platform. I’ve never really accepted “web apps” as a viable solution compared to native apps, but I understand that it’s better to have something working on all targets than to be mired in learning Java, Objective C, and C# before writing and debugging the same application three times. I may think that HTML is twice as hard and convoluted as app development should be, but that’s still 50% better than that last sentence (which was also difficult and convoluted). If you’re targeting all the platforms it’s nirvana to limit the rewrite to just the part that needs to change: the user experience. Android and iOS are as legitimate development platforms as desktop these days and the world cries out for a high-quality native app toolkit that crosses these boundaries. Today’s news has not dampened my enthusiasm about Qt and QML one bit. There is however one thing that my inner arm-chair general would like to expound upon, at length…

Widgets Rant

On the whole I am pleased with Digia’s stated commitment to widgets. I just hope that Digia doesn’t think widgets are the same cross-platform solution that they were for ye olde Trolltech. Widgets are great for certain UIs, UIs that are a bit out of fashion these days but still quite effective. I’m glad to hear they will receive loving care again (at least, I got that impression from the Qt blog post). But they quickly run into limitations on embedded or mobile UIs. The Brisbane office ran into this with Qtopia, which was an attempt to take Qt and Qt widgets and make an appealing mobile stack with it. Let me delineate those limitations from my perspective, this comes primarily from my experience on Qtopia way back in 2007:

1. Expensive
   I’ll start with the example that stands out to me in this area. One type of widget that seems to be missed greatly in QML are the layouts. They are really convenient, they use the space well and lay things out easily and with parameterized control. You can use them everywhere, really convenient, great for desktop UIs. But their awesome power comes at a terrible cost, let’s use cpp file size as a proxy for implementation complexity and performance:
   
wc -l src/widgets/kernel/q*layout*.cpp
8593 total
wc -l src/quick/items/qquickpositioners.cpp
1699 items/qquickpositioners.cpp

   q*layout*.cpp was used to catch all the various parts involved like qlayout.cpp, qlayoutengine.cpp, qlayoutitem.cpp and the various different layout implementations like Box, Stacked, Grid and Form layouts. But those three backend files alone are larger than qquickpositioners.cpp, which contains both the infrastructure base class and the implementations of the four positioner types (they’re so tiny, it barely makes sense to put them in their own files). Positioners do less, that’s why they’re called Positioners. They only position, they do no expensive cooperative resizing. But they also do more in that they have animation capabilities beyond those of the widget layouts because that’s also cheap with QML (the widget backend isn’t as amenable to animation as QML).

   Layouts vs Positioners is a representative sample of the performance implications of widgets vs primitives. The QtQuick primitives only do the cheapest stuff, so it’s not that expensive to throw them around everywhere. Its also pretty cheap to animate their properties and reevaluate the layout code for every single frame. The widgets try to do everything, so they are super-convenient but just lag out if you hit hardware performance constraints. Hint: You regularly run into those on mobile. If you want it to be as smooth as velvety butter, like everyone does these days, you have to consider the performance of your UI components and trim the unnecessary functionality on a per UI basis.

   Perhaps it’s because I wrote most of them, but the positioners are where I always think of this problem. Half the time I use a Column I want it to just do my sizing for me like a VBoxLayout. But then I remember that the same Column{} might then be used in a ListView delegate and have thirty instantiated every second while it tries to scroll smoothly. I can’t screw up that vital use-case because some other times I might use that functionality. Even a default off boolean flag would harm that usecase, because we want the performance to ‘just work’ for developers who use QtQuick. Adding the boolean means people will turn on the boolean, then use it in a delegate, then spend a day trawling the docs trying to figure out why their scrolling is no longer smooth. Then they’ll rewrite it hastily with the boolean off because they’re annoyed. Note that in that specific case, a second set of elements where you choose to incur that cost would work. But no-one has written them yet, and they can’t replace the QtQuick primitives.

2. Platform Specific

   This claim may sound crazy at first. I can already hear people screaming that widgets are the core of Qt’s cross-platform solution. Well that’s true – for desktops. The three desktop platforms all run on practically the same hardware. Mac may try to throw in as many arbitrary differences as it can, but it’s still fundamentally the same hardware, ergo same hardware user interface. For a screen that big, with a mouse and keyboard, certain UI elements and general designs work. This is especially true with the convention that windows don’t take the whole screen and can re-size fluidly (in rectangles). This is what widgets were made for, buttons that are ‘pressed’ when you hit space bar or click. Combo boxes with text completion and scrollbars. Works great on all desktop platforms, not originally designed for a tiny touch screen. And even if it was, it would have been a resistive touch screen not capacitive. That alone makes a big difference, as it requires big buttons for touch friendly UIs whereas resistive screens work with a stylus approximating a mouse (especially with a hardware keyboard). You get the idea, the existing widget set is sub-optimal for devices. It can’t really be extended either – because that means letting mobile-optimized widgets creep into your desktop app and then that’s the version that sucks.

   This could be solved with significantly different implementations per widget, per platform. This is terribly expensive in resources to build and maintain, and means you’ll never know just how your app will look on another platform. That solution also makes widgets even more expensive in terms of performance. This is the approach that Qt Quick Components were originally aiming for, except that by using QML and separate implementations it’s feasible for each platform to have an efficient and effective implementation.

   The size of the screen, the different input device, the fact you have your thumb obscuring anything you try to press, mobiles are different devices in terms of input. They are not as similar as the desktop platforms and this makes the cross-platform UI idea exponentially harder. Speaking to Mac developers I’ve heard that Qt on Mac still hasn’t managed to make UIs that fit in well despite it being the same hardware! This is why for A-Grade applications I’ve given up on the cross-platform UI approach. It’s easier to just have a great, flexible UI toolkit where the UI layer can be easily decoupled from the (truly portable) logic. It’s the obvious way to get pixel perfect results on time, every time.

3. Inflexible

   Mobile UIs are much more likely to have custom UI styling instead of trying to ‘look native’. Maybe it’s just the proliferation of casual games, or the fact that I still do all my work with my ivory tower box. Or perhaps its because embedded platforms and fledgling mobile platforms don’t have a pre-existing native style baked into the widget classes. Styling a widget for an entirely custom look? You can use style sheets to tweak parameters, or write your own QStyle, but that’s not easy. The easy approach is to just reimplement it yourself, a MyButton widget that you paint manually. Widgets were made to look native, not look like you want them too, until you write custom ones.

   This is why QtQuick offers primitives, not widgets. The QtQuick Components project can make widget like controls with QtQuick, when they have the time. They’ve already done it for 3-5 platforms (how many does the Desktop components count for?) which I’m aware of. These primitives are the right level of item from which to build a custom UI, whether that’s for a custom looking UI or just to write the platform styled controls. That extreme flexibility may hurt it in the short term while there are no controls for desktop shipped with Qt. But if you want something different than what the standard widgets provide, QML makes it a lot easier now (and just use widgets if they do what you want!).

Those are the three main issues I saw trying to use QtWidgets on mobile, and the ones that convinced me that QML (or another such approach) is the future for mobile UIs.

I want to see widgets working on embedded platforms, as they are still the most expedient method of building a cross-platform UI. I had some widget filled applications on the N900, they were plentiful and functional. They were hideous eyesores in terms of aesthetic design, but you can’t have everything. You couldn’t easily make a great looking mobile app with widgets for the reasons described above. The quick and dirty apps work like a charm and I don’t deny they’re an important usecase. If I was just trying to get an app where some buttons triggered a backend, but across all my devices at home(I currently have three Nokia devices, so three separate platforms there, two major versions of Android, plus Linux and Windows PCs) I’d use widgets[1]. But if you want an app that’s smoothly animated, fits the device perfectly, and has your own custom UI? You may as well write a MyWidget that manages the whole thing internally for all the help QWidget provides; seriously, that’s your best bet. QML is lighter weight, more flexible, and based off primitives that aren’t associated with any platform.

Of the Previous Chapter…

Since I don’t like living in the past, I’ll tack my quick Trolltech/Nokia retrospective on the end here.

I only joined Trolltech less than a year before it was acquired. I spent that time first as an intern on Qtopia and then wrote the Qtopia theme editor. The Nokia acquisition was perfectly timed for me – I had just finished the theme editor and passed it to QA. I got to do all the fun part of developing it without any boring maintenance :D. I never understood why Nokia didn’t want Qtopia, but with that cancelled we moved onto developing the first iteration of QML. It was a great research project, trying to figure out the best way to write mobile UIs that don’t need to hack the QApplication event filter. Unfortunately, our mandate was actually to just write the next version of .ui files. We had overstepped the bounds quite a bit, and eventually got cancelled for that (we got champagne as a consolation prize but we still haven’t drunk it). The month after we were cancelled was also a lot of fun, as we were encouraged to learn new things while we waited for another project. I learned OpenGL, and how to cook a gourmet breakfast for the entire team. Eventually we were un-cancelled because we had something working and it turns out that’s pretty valuable. It wasn’t long before QML began to mature to a demonstrable level:

The blinking text isn’t post-processing and is included in the code line count.

It’s around this point that I went to Oslo for eight months, because that’s long enough to justify the 30 hour journey each way. It was a great office, and not just because they reopened my eyes to the joy of Starcraft. Far away from my home office, I spent most of my time evangelizing QML and growing a mustache (I think there’s one guy who now has those two permanently associated in his mind). Jeg også lært litt norsk, og å langrennski.

I returned to Brisbane in time to work on QtQuick 2.0, and got a prize assignment. The most essential part of a modern, fluid UI is the particle system and we only had a labs level particle system in Qt Quick 1. For QtQuick 2.0 to be taken seriously, it needed a world-class particle system. So that’s what I wrote for the next year. It’s powerful, versatile and hardware accelerated, but along the way I realized just how few use-cases we had. Those we had were being performed admirably, but my inspiration far too often turned to graphics scene tech demos; which is just the wrong league. Once QtQuick 2 starts being used in actual UIs I hope to fine-tune the particle system towards where it can provide the most convenient aesthetic punch. Recently I’ve been noticing particles in more games (Bastion has wonderful particles), so perhaps I can use that to refine my understanding of the use-cases.

Then we worked on a secret project for a while. While interesting, I tend to prefer work that I can talk about openly. In the [redacted] project, we did some great stuff. We [redacted]. [redacted] QML [redacted]. We even managed to overcome [redacted] and against all odds [redacted]. And I worked on cool [redacted] with [redacted]! It was fun but it’s a bit depressing that everything I say about it will just end up redacted. Especially since you can read the whole story on Engadget – alas that doesn’t lift my obligation to secrecy.

We finally got a UI designer to work with in 2012, but by then it was too late. Existing demos were revitalized with fresh and mobile-styled designs, but we didn’t have time for research. At least SameGame evolved from the 300 LoC toy into a full game with four game modes including a set of QML levels I designed[2]. I also like the one time that I was wrote a custom shader to get the specific effect the designer wanted (in tweetsearch), a shame we didn’t do anything like that with particles. So that’s why the new Qt 5 demos don’t look like I just re-used the same bear-with-umbrella graphic everywhere. And if they do, at least the bear-with-umbrella looks better than the graphic it replaced: an animated smiley face I drew pixel by pixel.

There’s the abridged version of my Trolltech/Nokia story. I left out the stuff like the christmas party weekend at the beach resort, or the office response to my singing Gilbert and Sullivan (hint: they preferred my rendition of Jonathan Coulton’s work). I’m sure older trolls will provide all the colorful details and drinking stories. As for me I’ll just take my armor and ride along, singing from “Man of La Mancha”: And the wild winds of fortune will carry me onwards, o’ wither so ever they blow.

[1]Okay, “I” wouldn’t use widgets. I’d use QML because I just like QML, and have even been referred to as Mr. QML on occasion. But I’d honestly advise other people to use widgets.
[2]Designed is a bit strong a word for levels that I threw together in one afternoon. I still haven’t managed to clear level ten, but I assume someone with more time to devote to testing it would be able to 😉 .

Edit: Since I only started writing about QML on this blog recently, I should add to the retrospective that I used to write blog posts for Nokia: http://labs.qt.nokia.com/author/aalpert/. I also wrote one Qt Quarterly article, http://qt-project.org/quarterly/view/qt_quick_quits_quiescence .