Jusletter IT

This paper is concerned with a critical evaluation of three widely used Czech legal information retrieval systems: ASPI, Beck-online and Codexis. The legal information retrieved from these systems shape the existing Czech legal practice. Thus, the issue of how reliable their results are when compared to each other is of utmost importance. The paper reports results of an experiment aiming to frame the discussion on information retrieval from the Czech systems. The experiment uses relation-based searches to create three sets of court decisions. The union of these sets establishes ideal databases for individual research tasks that allow for a comparison of the Recall and the Precision of the individual systems. To assess relevance of documents in a system-independent way, three lawyers (two experts and a non-expert) provided for a binary coding of the relevance of the individual court decisions.