kidder's blog

A Small Recap

My intent in this project was to improve the state of the nepomuksearch:/ kioslave. Specifically, I wanted to add the features required to make desktop search a better experience in KDE. I also wanted to work on a more accessible GUI for desktop search. About halfway through the project I blogged about my work to date.

The State of KDE Search

I’ve introduced several new features since I’ve last blogged about my project.

• Date parsing has been improved. In particular relative dates are now supported (although this still needs work. More about that in a bit!)
• SPARQL queries are now supported directly in the kioslave. This is great if you want to show the results of a SPARQL query directly in a UI element – you simply create a KDirModel and point it to a nepomuksearch:/ URL. This was implemented directly as a result of a request from Alessandro Sivieri, who is using this in his GSoC project.
• Folder limiting is supported: it is now possible to specify where (and where not) to search, when searching for files.
• “Pretty sizes” are supported through the kioslave, so you can search by the size 7MB instead of just 7000000.
• Other minor improvements.

As well as improvements to the backend kioslave, I have also started on a UI for search.

In progress search client

This is the beginning of what will be integrated into Dolphin in time for 4.4 It is possible to search via full-text, as well as add various criteria that will be searched. The “Save This” button saves the search into the kioslave config file for later access.

Wrap Up and Thoughts

GSoC has been a great experience for me and I encourage any students interested in KDE, or open source in general, to consider applying next year. KDE has been a great project to work with, with great libraries and code, but most importantly great people to work with. I’d like to thank Sebastian Trüg most of all for his help as my mentor, and for putting up with me through the summer.

I consider my project mostly successful, although there were a few more things I would have liked to implement. One of the features that didn’t make it in time was automatic refreshing of relative date queries: so a date modified query with a time of “one hour ago” will update each hour to remain accurate. I had planned to implement this before the end of GSoC but it ended up being more complex than I originally thought.

The Future

I fully intend to remain developing for KDE after GSoC. I’m taking a short break to mark the end of GSoC, and then I will be back developing in KDE.

My immediate plans are to finish a few of the backend issues in the kioslave, and then merge this branch back into trunk. I’ll also continue working with Peter Penz on getting the UI code integrated into Dolphin in time for 4.4. Later on, I’d like to work on a more semantic-focused search UI, perhaps using the filter model developed by Alessandro.

I have been neglecting this blog ever since GSoC started, but I think I will finally sit down and write about my progress. Also, if you read the entire post, you may find some screenshots at the end.

Progress So Far.

So far I have only been working on backend-type code for the nepomuksearch:/ protocol. I have finished some important features that were missing, however.

Custom Default Search Queries

As an easy introduction to the world of KDE programming, I changed the hardcoded default search folders to be configurable in .kde/share/config/kionepomukuserqueriesrc (now that’s a mouthful!) So now instead of navigating to nepomuksearch:/ and seeing this:

You will see something like this:

where the config file looks like this:

[Searches]
All searches=tag_nepomuk

[tag_nepomuk]
Query=hasTag:nepomuk

In the future this config file will allow you to specify additional options per default search.

Negation Queries

Next I implemented various improvements to the query parser and query service. It is now possible to use negative queries, both in the Lucene full-text index, and in SPRAQL requests. For example, I can search for documents containing the tag nepomuk but not the tag soprano with a query like this:

nepomuksearch:/hasTag:nepomuk -hasTag:soprano

Similarly, it is possible to search for resources that are not rated above a certain value.

nepomuksearch:/-numericRating>8

Of course, this is pretty silly, as this is equal to using <= (in fact, in the SPARQL generation, this is exactly what happens). It is also possible to use negation queries in full-text queries with the same syntax:

nepomuksearch:/hasTag:nepomuk -gsoc

This will return all resources tagged with the tag nepomuk without “gsoc” in any indexed field. Of course, there is a caveat: Lucene only allows queries with at least one positive term, so only specifying only a negative term will give you zero results, rather than all resources not containing that term. This is not generally a problem – only specifying a negative term will return many many results if your entire home folder is indexed, which is not generally very useful.

Date Parsing

Up until now, there has been no method to specify dates in a nepomuksearch:/ query. This means there was no possible way to say “show me all resources modified after yesterday.” I’ve built upon (read: shamelessly stolen ) some work that Sebastian Trüg has done for Scribo to implement a generic date/time parsing framework for use in Nepomuk. This means queries such as

nepomuksearch:/sourceModified>"July 26"

will now work properly. This is still quite raw, and will be seeing many new things soon. One of the features I’m working on is support for relative dates and times – that is, instead of specifying a date such as “June 30″, you can say “yesterday”. This will allow you to make a virtual folder containing all resources changed after yesterday that will always stay updated. Wait a minute, you might say, this is already possible – one of the default search folders is “Yesterday’s Files”. These default folders are hardcoded, however, and there is no method to make a query like this through a nepomuksearch:/ query string.

Next Steps

While there is still some backend type code to work on, I’m starting to working on the new UI for search. Here is my first UI mockup:

The idea is that when the user types a query into the search box, the advanced search pane drops down. The user can select a search criterion in the first combo box, and an appropriate set of relations will appear in  the second. The third field will contain an appropriate input field for the type of date: for tags, a button to bring up a tag cloud, for dates, a date widget that brings up a calendar, etc. Users can press the “+” and “-” buttons to add and remove criteria to the search.

Instead of a breadcrumb type display for the location bar, Dolphin will simply display “Custom Search  Query”. Upon a click, just like in the breadcrumb display, this will change to show the actual URI.

Any thoughts, improvements, suggestions, flames? Please post them in the comments! This will also be cross-posted to kde-usability.

One of the things that I found helpful in writing my proposal was to look at succesful proposals from previous years. This helped me gain an appreciation for the depth and breadth utilized in previous accepted proposals. On this note, I’d like to post the proposal I used for 2009 – Improving Search and Virtual Folders in KDE4.

Of course, this may also be useful to people who are just interested in my project this summer.

This year I have been accepted for the Google Summer of Code, for the mentoring organization KDE. I am incredibly excited to get to know the folks from KDE (my application abstract is here) – from my experiences reading their blogs and other materials they seem to be a very close and friendly community.

My project’s primary proposal is to implement what I see as one of the biggest deficiencies in KDE – the lack of a full-featured search feature. Being my preferred desktop environment, it was slightly disappointing to see this left undone. A little searching, however, revealed that all of the backend code is actually there – it is merely some front-end work that needs to be accomplished. This got my very worked up for a bit as I realized what I could do to remedy this situation – and so I submitted my application for GSoC 2009. I think it is a great thing that I have the opportunity to contribute to my most-used and favorite open source project, and I’d like to thank all the nice folks at KDE and Google for making this happen.