My intent in this project was to improve the state of the nepomuksearch:/ kioslave. Specifically, I wanted to add the features required to make desktop search a better experience in KDE. I also wanted to work on a more accessible GUI for desktop search. About halfway through the project I blogged about my work to date.

The State of KDE Search

I’ve introduced several new features since I’ve last blogged about my project.

• Date parsing has been improved. In particular relative dates are now supported (although this still needs work. More about that in a bit!)
• SPARQL queries are now supported directly in the kioslave. This is great if you want to show the results of a SPARQL query directly in a UI element – you simply create a KDirModel and point it to a nepomuksearch:/ URL. This was implemented directly as a result of a request from Alessandro Sivieri, who is using this in his GSoC project.
• Folder limiting is supported: it is now possible to specify where (and where not) to search, when searching for files.
• “Pretty sizes” are supported through the kioslave, so you can search by the size 7MB instead of just 7000000.
• Other minor improvements.

As well as improvements to the backend kioslave, I have also started on a UI for search.

In progress search client

This is the beginning of what will be integrated into Dolphin in time for 4.4 It is possible to search via full-text, as well as add various criteria that will be searched. The “Save This” button saves the search into the kioslave config file for later access.

Wrap Up and Thoughts

GSoC has been a great experience for me and I encourage any students interested in KDE, or open source in general, to consider applying next year. KDE has been a great project to work with, with great libraries and code, but most importantly great people to work with. I’d like to thank Sebastian Trüg most of all for his help as my mentor, and for putting up with me through the summer.

I consider my project mostly successful, although there were a few more things I would have liked to implement. One of the features that didn’t make it in time was automatic refreshing of relative date queries: so a date modified query with a time of “one hour ago” will update each hour to remain accurate. I had planned to implement this before the end of GSoC but it ended up being more complex than I originally thought.

The Future

I fully intend to remain developing for KDE after GSoC. I’m taking a short break to mark the end of GSoC, and then I will be back developing in KDE.

My immediate plans are to finish a few of the backend issues in the kioslave, and then merge this branch back into trunk. I’ll also continue working with Peter Penz on getting the UI code integrated into Dolphin in time for 4.4. Later on, I’d like to work on a more semantic-focused search UI, perhaps using the filter model developed by Alessandro.

Progress So Far.

So far I have only been working on backend-type code for the nepomuksearch:/ protocol. I have finished some important features that were missing, however.

Custom Default Search Queries

As an easy introduction to the world of KDE programming, I changed the hardcoded default search folders to be configurable in .kde/share/config/kionepomukuserqueriesrc (now that’s a mouthful!) So now instead of navigating to nepomuksearch:/ and seeing this:

You will see something like this:

where the config file looks like this:

[Searches]
All searches=tag_nepomuk

[tag_nepomuk]
Query=hasTag:nepomuk

In the future this config file will allow you to specify additional options per default search.

Negation Queries

Next I implemented various improvements to the query parser and query service. It is now possible to use negative queries, both in the Lucene full-text index, and in SPRAQL requests. For example, I can search for documents containing the tag nepomuk but not the tag soprano with a query like this:

nepomuksearch:/hasTag:nepomuk -hasTag:soprano

Similarly, it is possible to search for resources that are not rated above a certain value.

nepomuksearch:/-numericRating>8

Of course, this is pretty silly, as this is equal to using <= (in fact, in the SPARQL generation, this is exactly what happens). It is also possible to use negation queries in full-text queries with the same syntax:

nepomuksearch:/hasTag:nepomuk -gsoc

This will return all resources tagged with the tag nepomuk without “gsoc” in any indexed field. Of course, there is a caveat: Lucene only allows queries with at least one positive term, so only specifying only a negative term will give you zero results, rather than all resources not containing that term. This is not generally a problem – only specifying a negative term will return many many results if your entire home folder is indexed, which is not generally very useful.

Date Parsing

Up until now, there has been no method to specify dates in a nepomuksearch:/ query. This means there was no possible way to say “show me all resources modified after yesterday.” I’ve built upon (read: shamelessly stolen ) some work that Sebastian Trüg has done for Scribo to implement a generic date/time parsing framework for use in Nepomuk. This means queries such as

nepomuksearch:/sourceModified>"July 26"

will now work properly. This is still quite raw, and will be seeing many new things soon. One of the features I’m working on is support for relative dates and times – that is, instead of specifying a date such as “June 30″, you can say “yesterday”. This will allow you to make a virtual folder containing all resources changed after yesterday that will always stay updated. Wait a minute, you might say, this is already possible – one of the default search folders is “Yesterday’s Files”. These default folders are hardcoded, however, and there is no method to make a query like this through a nepomuksearch:/ query string.

Next Steps

While there is still some backend type code to work on, I’m starting to working on the new UI for search. Here is my first UI mockup:

The idea is that when the user types a query into the search box, the advanced search pane drops down. The user can select a search criterion in the first combo box, and an appropriate set of relations will appear in  the second. The third field will contain an appropriate input field for the type of date: for tags, a button to bring up a tag cloud, for dates, a date widget that brings up a calendar, etc. Users can press the “+” and “-” buttons to add and remove criteria to the search.

Instead of a breadcrumb type display for the location bar, Dolphin will simply display “Custom Search  Query”. Upon a click, just like in the breadcrumb display, this will change to show the actual URI.

Any thoughts, improvements, suggestions, flames? Please post them in the comments! This will also be cross-posted to kde-usability.

My project this summer is to do a little work to improve the lacking desktop search, via a relatively new technology in KDE: Nepomuk. Nepomuk allows us to do many cool things, but the big part I’m concerned with is the ability to do full-text indexed search. I plan to do both some UI improvements, mostly in Dolphin, and some changes to the nepomuksearch kio_slave to allow for greater usability. My full proposal is available here.

I’ll be posting here regularly with details of my project, as well as calls for input on the desktop search interface.

Of course, this may also be useful to people who are just interested in my project this summer.

My project’s primary proposal is to implement what I see as one of the biggest deficiencies in KDE – the lack of a full-featured search feature. Being my preferred desktop environment, it was slightly disappointing to see this left undone. A little searching, however, revealed that all of the backend code is actually there – it is merely some front-end work that needs to be accomplished. This got my very worked up for a bit as I realized what I could do to remedy this situation – and so I submitted my application for GSoC 2009. I think it is a great thing that I have the opportunity to contribute to my most-used and favorite open source project, and I’d like to thank all the nice folks at KDE and Google for making this happen.

I use subversion for source control, so it seems natural to use subversion version numbers to automatically add a version string. This turns out to have a couple of subtle issues:

• I need one version number for the whole program, not individual version numbers for each file
• I want to see if the code built has been modified since the last commit

Now I did a bit of internet searching, and came across svnversion! This seems to solve both of my problems quite nicely – it will output a version number for a directory, and it can append ‘M’ to the end of the version string if the working copy was modified. This even seems to be cross platform! The only thing left to do is to make a script to embed this information into a source file, and then add this script to out build system.

However, I’ve neglected a couple of missing things that could be useful:

• what if I want the revision number as an integer?
• what if I want the modified flag as a boolean?
• what if I want the date the application was built?

These are all minor issues – nothing some perl-fu can’t solve. It would still be nice if I didn’t have to write all of this myself (as laziness is the foundation of efficiency).

Lucky for me, I don’t. There is a nice program called svnrev that will embed all of this information into a C++ or Java source file. It’s cross-platform and even comes with a Linux and Windows binary. Now I just needed to embed all of this into my build scripts.

I was only able to solve this problem satisfactorally when I started using CMake for builds. Before, I had been building on Linux and Windows, and had to maintain both an autotools configuration and Visual Studio solution. Switching to CMake gave me an easy way to generate native builds for each platform (as well as solve my versioning problems!) I now have a custom target in my CMakeLists.txt that my main target depends on. This looks something like this:

IF(UNIX)

  ADD_CUSTOM_TARGET(svnversion ALL ./svn_update.sh WORKING_DIRECTORY ${SOURCE_DIR}/tools/)

ENDIF(UNIX)

IF(WIN32)

  ADD_CUSTOM_TARGET(svnversion ALL ./svn_update.bat WORKING_DIRECTORY ${SOURCE_DIR}/tools/)

ENDIF(WIN32)

svn_update.{bat,sh} is a simple shell script that calls svnrev on all of my source files. I then add the svn_update script and svnrev to a tools/ directory in subversion, as well as a CMakeLists.txt like the one above. In my main CMake target, I just call

ADD_DEPENDENCIES(target svnversion)

(defun reload-file ()
  (interactive)
  (let ((curr-scroll (window-vscroll)))
    (find-file (buffer-name))
    (set-window-vscroll nil curr-scroll)
    (message "Reloaded file")))
 
(global-set-key "\C-c\C-r" 'reload-file)

Put that in .emacs, and you simply have to type C-c C-r to reload the current file (while saving your scroll position!)