Archive for März, 2013

Release 1.4: „The Search Progess Bar Has Disappeared“

Montag, März 18th, 2013

We published the new main release 1.4


When you do a peer-to-peer search, the requesting peer must wait for the remote peers to submit their remote search result. To show a progress of remote searches, the search interface had a progress bar. It’s still there, but when we showed the new YaCy Release 1.4 at the Linuxtage Fair in Chemnitz last weekend, people said: „is there a bug? the progress bar does not show„. No, it’s not a bug, in many cases the bar flashes so fast that you cannot see the bar any more.


Furthermore, the search result quality has increased. This is the result of the advancing deep Solr integration not only in local search, but also in remote search. The integration is not yet fully finished, but it now shows a new quality of integration flexibility, speed and relevancy of search results.

Here are some more details about the main changes in YaCy 1.4:

  • This release includes mainly a deeper Solr integration, much more Solr fields are filled, Solr has now mutli-core capabilities and a second core with a webgraph was added (but deactivated for further testing).
  • The opensearch result writer of the integrated solr has now all the features as the original opensearch result servlet of YaCy had, and the file search interface „yacyinteractive“ now uses this new result writer instead the old one. The search of that interface is now much faster.
  • The default search process has undergone a full re-design and a lot of testing was done to fix problems with the to-solr migration. The normal (local) search is now very fast, especially in portal mode and even in p2p mode.
  • The ranking was strongly enhanced, there is now a support for flexible field boosts, boost functions and boost queries (see servlet /RankingSolr_p.html). All these ranking functions had been made editable and there is a new configuration sevlet for this. Furthermore, there are several ranking
    schemas predefined, one for default internet search, one for sort-by-date and one for intranet search requests, which is triggered automatically if a site-operator is used. Intranet search ranking rates deep links higher than shallow which returns more specific document types. Remote searches are done using the local ranking profile, not the remote profile.
  • The selection of target peers had been enhanced, now all robinson peers which have a solr interface are searched using that interface rather with the old YaCy interface.
  • There should also an enhancement in indexing speed as there are less requests to the solr for
    doing that and index updates are bundled together while forced commits had been reduced using a new solr 4.1 soft-commit feature.
  • There had also been fixes to some memory leaks and the overall memory usage should be lower.
  • There is also a large number of small bug fixes.