Patent retrieval: a literature review
---
paper_title: Our Divided Patent System

paper_content:

In this comprehensive new study, we evaluate all substantive decisions rendered by any court in every patent case filed in 2008 and 2009 — decisions made between 2009 and 2013. We assess the outcome of litigation by technology and industry. We relate the outcomes of those cases to a host of variables, including variables related to the parties, the patents, and the courts in which those cases were litigated. We find dramatic differences in the outcomes of patent litigation by both technology and industry. For example, owners of patents in the pharmaceutical industry fare much better in dispositive litigation rulings than do owners of patents in the computer & electronics industry, and chemistry patents have much greater success in litigation than their software or biotech counterparts. Our results provide an important window into both patent litigation and the industry-specific battles over patent reform. And they suggest that the traditional narrative of industry-specific patent disputes, which pits the IT industries against the life sciences, is incomplete.
---
paper_title: Current Challenges in Patent Information Retrieval

paper_content:

Intellectual property in the form of patents plays a vital role in today's increasingly knowledge-based economy. This book assembles state-of-the art research and is intended to illustrate innovative approaches to patent information retrieval.
---
paper_title: CLEF-IP 2011: Retrieval in the Intellectual Property Domain

paper_content:

The patent system is designed to encourage disclosure of new technologies and novel ideas by granting exclusive rights on the use of inventions to their inventors, for a limited period of time. Before a patent can be granted, patent o ces around the world perform thorough searches to ensure that no previous similar disclosures were made. In the intellectual property terminology, such kind of searches are called prior art searches. In some industries, the number of granted patents a company owns has a high impact on the market value of the company. This underlines the importance of well-performed prior art searches. Together with the Trec Chem track [5], also organized by our institution, the Clef Ip e ort comes to complete the work that is being done in the series of Ntcir workshops (see for example [4]). The rst Clef Ip track ran within Clef 2009. The purpose of the track was twofold: to encourage and facilitate research in the area of patent retrieval by providing a large clean data set for experimentation; to create a large test collection of patents in the three main European languages for the evaluation of cross lingual information access. The Clef Ip data set includes documents published by the European Patent O ce (Epo) which contain a mixture of English, German and French content. The track focused on the task of prior art search. In 2010 and 2011, the Clef Ip track was organized as a benchmarking activity (lab) in the Clef conference. In these years, the main goal of the Clef Ip e ort remained the same to foster research in the patent retrieval area, and provide a large clean data set. To this end, the number of tasks in the track was increased and the data set was enlarged. Recognizing the importance of patent classi cations in the daily activity of an intellectual property professional, in 2010 the Clef Ip benchmarking activity included a patent classi cation task. The participants were asked to classify
---
paper_title: Modern Information Retrieval

paper_content:

From the Publisher: ::: This is a rigorous and complete textbook for a first course on information retrieval from the computer science (as opposed to a user-centred) perspective. The advent of the Internet and the enormous increase in volume of electronically stored information generally has led to substantial work on IR from the computer science perspective - this book provides an up-to-date student oriented treatment of the subject.
---
paper_title: Understanding the Realities of Modern Patent Litigation

paper_content:

Sixteen years ago, two of us published the first detailed empirical look at patent litigation. In this Article, we update and expand the earlier study with a new hand-coded data set. We evaluate all substantive decisions rendered by any court in every patent case filed in 2008 and 2009 — decisions made between 2009 and 2013. We consider not just patent validity but also infringement and unenforceability. Moreover, we relate the outcomes of those cases to a host of variables, including variables related to the parties, the patents, and the courts in which those cases were litigated. The result is a comprehensive picture of the outcomes of modern patent litigation, one that confirms conventional wisdom in some respects but upends it in others. In particular, we find a surprising amount of continuity in the basic outcomes of patent lawsuits over the past twenty years, despite rather dramatic changes in who brought patent suits during that time.
---
paper_title: Wikipedia-based query phrase expansion in patent class search

paper_content:

Relevance feedback methods generally suffer from topic drift caused by word ambiguities and synonymous uses of words. Topic drift is an important issue in patent information retrieval as people tend to use different expressions describing similar concepts causing low precision and recall at the same time. Furthermore, failing to retrieve relevant patents to an application during the examination process may cause legal problems caused by granting an existing invention. A possible cause of topic drift is utilizing a relevance feedback-based search method. As a way to alleviate the inherent problem, we propose a novel query phrase expansion approach utilizing semantic annotations in Wikipedia pages, trying to enrich queries with phrases disambiguating the original query words. The idea was implemented for patent search where patents are classified into a hierarchy of categories, and the analyses of the experimental results showed not only the positive roles of phrases and words in retrieving additional relevant documents through query expansion but also their contributions to alleviating the query drift problem. More specifically, our query expansion method was compared against relevance-based language model, a state-of-the-art query expansion method, to show its superiority in terms of MAP on all levels of the classification hierarchy.
---
paper_title: A Methodology for Building a Patent Test Collection for Prior Art Search

paper_content:

This paper proposes a methodology for the construction of a patent test collection for the task of prior art search. Key to the justification of the methodology is an analysis of the nature and structure of patent documents and the patenting process. These factors enable a corpus of patent documents to be reverse engineered in order to arrive at high quality, realistic, relevance assessments. The paper first outlines the case for such a prior art search test collection along with the characteristics of patent documents, before describing the proposed method. Further research and development will be directed towards the application of this methodology to create a suite of prior art search topics for the evaluation of patent retrieval systems. We also include a preliminary analysis of its applica
---
paper_title: CLEF-IP 2011: Retrieval in the Intellectual Property Domain

paper_content:

The patent system is designed to encourage disclosure of new technologies and novel ideas by granting exclusive rights on the use of inventions to their inventors, for a limited period of time. Before a patent can be granted, patent o ces around the world perform thorough searches to ensure that no previous similar disclosures were made. In the intellectual property terminology, such kind of searches are called prior art searches. In some industries, the number of granted patents a company owns has a high impact on the market value of the company. This underlines the importance of well-performed prior art searches. Together with the Trec Chem track [5], also organized by our institution, the Clef Ip e ort comes to complete the work that is being done in the series of Ntcir workshops (see for example [4]). The rst Clef Ip track ran within Clef 2009. The purpose of the track was twofold: to encourage and facilitate research in the area of patent retrieval by providing a large clean data set for experimentation; to create a large test collection of patents in the three main European languages for the evaluation of cross lingual information access. The Clef Ip data set includes documents published by the European Patent O ce (Epo) which contain a mixture of English, German and French content. The track focused on the task of prior art search. In 2010 and 2011, the Clef Ip track was organized as a benchmarking activity (lab) in the Clef conference. In these years, the main goal of the Clef Ip e ort remained the same to foster research in the patent retrieval area, and provide a large clean data set. To this end, the number of tasks in the track was increased and the data set was enlarged. Recognizing the importance of patent classi cations in the daily activity of an intellectual property professional, in 2010 the Clef Ip benchmarking activity included a patent classi cation task. The participants were asked to classify
---
paper_title: CLEF-IP 2010: Retrieval Experiments in the Intellectual Property Domain

paper_content:

The CLEF-IP track ran for the first time within CLEF 2009. The purpose of the track was twofold: to encourage and facilitate research in the area of patent retrieval by providing a large clean data set for experimentation; to create a large test collection of patents in the three main European languages for the evaluation of cross-lingual information access. The track focused on the task of prior art search. The 15 European teams who participated in the track deployed a rich range of Information Retrieval techniques adapting them to this new specific domain and task. A large-scale test collection for evaluation purposes was created by exploiting patent citations.
---
paper_title: A study of query reformulation for patent prior art search with partial patent applications

paper_content:

Patents are used by legal entities to legally protect their inventions and represent a multi-billion dollar industry of licensing and litigation. In 2014, 326,033 patent applications were approved in the US alone -- a number that has doubled in the past 15 years and which makes prior art search a daunting, but necessary task in the patent application process. In this work, we seek to investigate the efficacy of prior art search strategies from the perspective of the inventor who wishes to assess the patentability of their ideas prior to writing a full application. While much of the literature inspired by the evaluation framework of the CLEF-IP competition has aimed to assist patent examiners in assessing prior art for complete patent applications, less of this work has focused on patent search with queries representing partial applications. In the (partial) patent search setting, a query is often much longer than in other standard IR tasks, e.g., the description section may contain hundreds or even thousands of words. While the length of such queries may suggest query reduction strategies to remove irrelevant terms, intentional obfuscation and general language used in patents suggests that it may help to expand queries with additionally relevant terms. To assess the trade-offs among all of these pre-application prior art search strategies, we comparatively evaluate a variety of partial application search and query reformulation methods. Among numerous findings, querying with a full description, perhaps in conjunction with generic (non-patent specific) query reduction methods, is recommended for best performance. However, we also find that querying with an abstract represents the best trade-off in terms of writing effort vs. retrieval efficacy (i.e., querying with the description sections only lead to marginal improvements) and that for such relatively short queries, generic query expansion methods help.
---
paper_title: Overview Of Patent Retrieval Task At NTCIR-3

paper_content:

This paper describes the Patent Retrieval Task in the Fourth NTCIR Workshop, and the test collections produced in this task. We perform the invalidity search task, in which each participant group searches a patent collection for the patents that can invalidate the demand in an existing claim. We also perform the automatic patent map generation task, in which the patents associated with a specific topic are organized in a multi-dimensional matrix.
---
paper_title: Overview of the Patent Retrieval Task at the NTCIR-6 Workshop

paper_content:

In the Sixth NTCIR Workshop, we organized the Patent Retrieval Task and performed three subtasks; Japanese Retrieval, English Retrieval, and Classification. This paper describes the Japanese Retrieval Subtask and English Retrieval Subtask, both of which were intended for patent-to-patent invalidity search task. We report the evaluation results of the groups participating in those subtasks.
---
paper_title: Understanding the Realities of Modern Patent Litigation

paper_content:

Sixteen years ago, two of us published the first detailed empirical look at patent litigation. In this Article, we update and expand the earlier study with a new hand-coded data set. We evaluate all substantive decisions rendered by any court in every patent case filed in 2008 and 2009 — decisions made between 2009 and 2013. We consider not just patent validity but also infringement and unenforceability. Moreover, we relate the outcomes of those cases to a host of variables, including variables related to the parties, the patents, and the courts in which those cases were litigated. The result is a comprehensive picture of the outcomes of modern patent litigation, one that confirms conventional wisdom in some respects but upends it in others. In particular, we find a surprising amount of continuity in the basic outcomes of patent lawsuits over the past twenty years, despite rather dramatic changes in who brought patent suits during that time.
---
paper_title: Overview of the TREC 2011 Chemical IR Track

paper_content:

Abstract : TREC 2009 was the first year of the Chemical IR Track, which focuses on evaluation of search techniques for discovery of digitally stored information on chemical patents and academic journal articles. The track included two tasks: Prior Art (PA) and Technical Survey (TS) tasks. This paper describes how we designed the two tasks and presents the official results of eight participating groups.
---
paper_title: TREC-CHEM: large scale chemical information retrieval evaluation at TREC

paper_content:

Over the past decades, significant progress has been made in Information Retrieval (IR), ranging from efficiency and scalability to theoretical modeling and evaluation. However, many grand challenges remain. Recently, more and more attention has been paid to the research in domain specific IR applications, as evidenced by the organization of Genomics and Legal tracks in the Text REtrieval Conference (TREC). Now it is the right time to carry out large scale evaluations on chemical datasets in order to promote the research in chemical IR in general and chemical Patent IR in particular. Accordingly, we organize a chemical IR track in TREC (TREC-CHEM) in order to address the challenges in chemical and patent IR. This paper describes these challenges and the accomplishments of the first year and opens up the discussions for the next year.
---
paper_title: The use of MMR, diversity-based reranking for reordering documents and producing summaries

paper_content:

This paper presents a method for combining ::: query-relevance with information-novelty in the context ::: of text retrieval and summarization. The Maximal ::: Marginal Relevance (MMR) criterion strives to reduce ::: redundancy while maintaining query relevance in ::: re-ranking retrieved documents and in selecting appropriate passages for text summarization. Preliminary results ::: indicate some benefits for MMR diversity ranking ::: in document retrieval and in single document summarization. ::: The latter are borne out by the recent results of the ::: SUMMAC conference in the evaluation of summarization ::: systems. However, the clearest advantage is demonstrated ::: in constructing non-redundant multi-document ::: summaries, where MMR results are clearly superior to ::: non-MMR passage selection.
---
paper_title: Data sources on patents, copyrights, trademarks, and other intellectual property

paper_content:

In this book chapter, we provide a roadmap of the sources of data on the various forms of intellectual property protection. We first explain what data is available about patents, copyrights, trademarks, and other types of intellectual property, and where to find it. Then we identify and analyze data sources specifically relating to intellectual property licensing and litigation, growing areas of research by scholars and lawyers.
---
paper_title: Evaluation of machine-learning protocols for technology-assisted review in electronic discovery

paper_content:

Abstract Using a novel evaluation toolkit that simulates a human reviewer in the loop, we compare the effectiveness of three machine-learning protocols for technology-assisted review as used in document review for discovery in legal proceedings. Our comparison addresses a central question in the deployment of technology-assisted review: Should training documents be selected at random, or should they be selected using one or more non-random methods, such as keyword search or active learning? On eight review tasks -- four derived from the TREC 2009 Legal Track and four derived from actual legal matters -- recall was measured as a function of human review effort. The results show that entirely non-random training methods, in which the initial training documents are selected using a simple keyword search, and subsequent training documents are selected by active learning, require substantially and significantly less human review effort (P
---
paper_title: The Market Value of Blocking Patent Citations

paper_content:

There is a growing literature that aims at assessing the private value of knowledge assets and patents. It has been shown that patents and their quality as measured by citations received by future patents contribute significantly to the market value of firms beyond their R&D stocks. This paper goes one step further and distinguishes between different types of forward citations patents can receive at the European Patent Office. While a patent can be cited as non-infringing state of the art, it can also be cited because it threatens the novelty of patent applications ('blocking citations'). Empirical results from a market value model for a sample of large, R&D-intensive U.S., European and Japanese firms show that patents frequently cited as blocking references have a higher economic value for their owners than patents cited for nonblocking reasons. This finding adds to the patent value literature by showing that different types of patent citations carry different information on the economic value of patents. The result further suggests that the total number of forward citations can be an imprecise measure of patent value.
---
paper_title: Predicting query performance

paper_content:

We develop a method for predicting query performance by computing the relative entropy between a query language model and the corresponding collection language model. The resulting clarity score measures the coherence of the language usage in documents whose models are likely to generate the query. We suggest that clarity scores measure the ambiguity of a query with respect to a collection of documents and show that they correlate positively with average precision in a variety of TREC test sets. Thus, the clarity score may be used to identify ineffective queries, on average, without relevance information. We develop an algorithm for automatically setting the clarity score threshold between predicted poorly-performing queries and acceptable queries and validate it using TREC data. In particular, we compare the automatic thresholds to optimum thresholds and also check how frequently results as good are achieved in sampling experiments that randomly assign queries to the two classes.
---
paper_title: CLEF-IP 2010: Prior Art Retrieval Using the Different Sections in Patent Documents

paper_content:

In this paper we describe our participation in the 2010 CLEF-IP Prior Art Retrieval task where we examined the impact of information in dierent sections of patent documents, namely the title, abstract, claims, description and IPC-R sections, on the retrieval and re-ranking of patent documents. Using a standard bag-of-words approach in Lemur we found that the IPC-R sections are the most informative for patent retrieval. We then performed a re-ranking of the retrieved documents using a Logistic Regression Model, trained on the retrieved documents in the training set. We found indications that the information contained in the text sections of the patent document can contribute to a better ranking of the retrieved documents. The ocial results have shown that among the nine groups that participated in the Prior Art Retrieval task we achieved the eigth rank in terms of both Mean Average Precision (MAP) and Recall.
---
paper_title: Automated Patent Categorization and Guided Patent Search using IPC as Inspired by MeSH and PubMed

paper_content:

Document search on PubMed, the pre-eminent database for biomedical literature, relies on the annotation of its documents with relevant terms from the Medical Subject Headings ontology (MeSH) for improving recall through query expansion. Patent documents are another important information source, though they are considerably less accessible. One option to expand patent search beyond pure keywords is the inclusion of classification information: Since every patent is assigned at least one class code, it should be possible for these assignments to be automatically used in a similar way as the MeSH annotations in PubMed. In order to develop a system for this task, it is necessary to have a good understanding of the properties of both classification systems. This report describes our comparative analysis of MeSH and the main patent classification system, the International Patent Classification (IPC). We investigate the hierarchical structures as well as the properties of the terms/classes respectively, and we compare the assignment of IPC codes to patents with the annotation of PubMed documents with MeSH terms. Our analysis shows a strong structural similarity of the hierarchies, but significant differences of terms and annotations. The low number of IPC class assignments and the lack of occurrences of class labels in patent texts imply that current patent search is severely limited. To overcome these limits, we evaluate a method for the automated assignment of additional classes to patent documents, and we propose a system for guided patent search based on the use of class co-occurrence information and external resources.
---
paper_title: Overview Of Patent Retrieval Task At NTCIR-3

paper_content:

This paper describes the Patent Retrieval Task in the Fourth NTCIR Workshop, and the test collections produced in this task. We perform the invalidity search task, in which each participant group searches a patent collection for the patents that can invalidate the demand in an existing claim. We also perform the automatic patent map generation task, in which the patents associated with a specific topic are organized in a multi-dimensional matrix.
---
paper_title: PRES: a score metric for evaluating recall-oriented information retrieval applications

paper_content:

Information retrieval (IR) evaluation scores are generally designed to measure the effectiveness with which relevant documents are identified and retrieved. Many scores have been proposed for this purpose over the years. These have primarily focused on aspects of precision and recall, and while these are often discussed with equal importance, in practice most attention has been given to precision focused metrics. Even for recall-oriented IR tasks of growing importance, such as patent retrieval, these precision based scores remain the primary evaluation measures. Our study examines different evaluation measures for a recall-oriented patent retrieval task and demonstrates the limitations of the current scores in comparing different IR systems for this task. We introduce PRES, a novel evaluation metric for this type of application taking account of recall and the user's search effort. The behaviour of PRES is demonstrated on 48 runs from the CLEF-IP 2009 patent retrieval track. A full analysis of the performance of PRES shows its suitability for measuring the retrieval effectiveness of systems from a recall focused perspective taking into account the user's expected search effort.
---
paper_title: Cumulated gain-based evaluation of IR techniques

paper_content:

Modern large retrieval environments tend to overwhelm their users by their large output. Since all documents are not of equal relevance to their users, highly relevant documents should be identified and ranked first for presentation. In order to develop IR techniques in this direction, it is necessary to develop evaluation approaches and methods that credit IR methods for their ability to retrieve highly relevant documents. This can be done by extending traditional evaluation methods, that is, recall and precision based on binary relevance judgments, to graded relevance judgments. Alternatively, novel measures based on graded relevance judgments may be developed. This article proposes several novel measures that compute the cumulative gain the user obtains by examining the retrieval result up to a given ranked position. The first one accumulates the relevance scores of retrieved documents along the ranked result list. The second one is similar but applies a discount factor to the relevance scores in order to devaluate late-retrieved documents. The third one computes the relative-to-the-ideal performance of IR techniques, based on the cumulative gain they are able to yield. These novel measures are defined and discussed and their use is demonstrated in a case study using TREC data: sample system run results for 20 queries in TREC-7. As a relevance base we used novel graded relevance judgments on a four-point scale. The test results indicate that the proposed measures credit IR methods for their ability to retrieve highly relevant documents and allow testing of statistical significance of effectiveness differences. The graphs based on the measures also provide insight into the performance IR techniques and allow interpretation, for example, from the user point of view.
---
paper_title: Modern Information Retrieval

paper_content:

From the Publisher: ::: This is a rigorous and complete textbook for a first course on information retrieval from the computer science (as opposed to a user-centred) perspective. The advent of the Internet and the enormous increase in volume of electronically stored information generally has led to substantial work on IR from the computer science perspective - this book provides an up-to-date student oriented treatment of the subject.
---
paper_title: Selecting good expansion terms for pseudo-relevance feedback

paper_content:

Pseudo-relevance feedback assumes that most frequent terms in the pseudo-feedback documents are useful for the retrieval. In this study, we re-examine this assumption and show that it does not hold in reality - many expansion terms identified in traditional approaches are indeed unrelated to the query and harmful to the retrieval. We also show that good expansion terms cannot be distinguished from bad ones merely on their distributions in the feedback documents and in the whole collection. We then propose to integrate a term classification process to predict the usefulness of expansion terms. Multiple additional features can be integrated in this process. Our experiments on three TREC collections show that retrieval effectiveness can be much improved when term classification is used. In addition, we also demonstrate that good terms should be identified directly according to their possible impact on the retrieval effectiveness, i.e. using supervised learning, instead of unsupervised learning.
---
paper_title: Query Terms Extraction from Patent Document for Invalidity Search

paper_content:

This paper describes our patent retrieval system participated in the NTCIR-5 Patent Retrieval Task, Document Retrieval Subtask. The main scope of our method is the appropriate query expansion to improve recall. We extracted query terms from the topic claim, and expanded query terms extracted from sentences explained in the patent document including the topic claim. The explanation sentences were extracted by the method based on pattern matching and by the method based on the longest common subsequence length.
---
paper_title: Query construction based on concept importance for effective patent retrieval

paper_content:

Patent retrieval is a long query task whose aim is to retrieve all documents related to patent applications. However, current approaches face with the term mismatch problem, leading to low retrieval performance. To deal with this issue, we propose a novel automatic query construction approach based on semantic concept importance for effective patent retrieval. In this approach, natural language processing techniques are firstly adopted to analyze patent long query inputs. Then, candidate query concepts are generated according to the concept features. Further, a concept importance-based query construction algorithm is presented to select the representative query concepts. Experimental results on the standard patent dataset demonstrate that our proposed approach can significantly outperform other state-of-art methods.
---
paper_title: Parsimonious language models for information retrieval

paper_content:

We systematically investigate a new approach to estimating the parameters of language models for information retrieval, called parsimonious language models. Parsimonious language models explicitly address the relation between levels of language models that are typically used for smoothing. As such, they need fewer (non-zero) parameters to describe the data. We apply parsimonious models at three stages of the retrieval process: 1) at indexing time; 2) at search time; 3) at feedback time. Experimental results show that we are able to build models that are significantly smaller than standard models, but that still perform at least as well as the standard approaches.
---
paper_title: CLEF-IP 2010: Prior Art Retrieval Using the Different Sections in Patent Documents

paper_content:

In this paper we describe our participation in the 2010 CLEF-IP Prior Art Retrieval task where we examined the impact of information in dierent sections of patent documents, namely the title, abstract, claims, description and IPC-R sections, on the retrieval and re-ranking of patent documents. Using a standard bag-of-words approach in Lemur we found that the IPC-R sections are the most informative for patent retrieval. We then performed a re-ranking of the retrieved documents using a Logistic Regression Model, trained on the retrieved documents in the training set. We found indications that the information contained in the text sections of the patent document can contribute to a better ranking of the retrieved documents. The ocial results have shown that among the nine groups that participated in the Prior Art Retrieval task we achieved the eigth rank in terms of both Mean Average Precision (MAP) and Recall.
---
paper_title: Building queries for prior-art search

paper_content:

Prior-art search is a critical step in the examination procedure of a patent application. This study explores automatic query generation from patent documents to facilitate the time-consuming and labor-intensive search for relevant patents. It is essential for this task to identify discriminative terms in different fields of a query patent, which enables us to distinguish relevant patents from non-relevant patents. To this end we investigate the distribution of terms occurring in different fields of the query patent and compare the distributions with the rest of the collection using language modeling estimation techniques. We experiment with term weighting based on the Kullback-Leibler divergence between the query patent and the collection and also with parsimonious language model estimation. Both of these techniques promote words that are common in the query patent and are rare in the collection. We also incorporate the classification assigned to patent documents into our model, to exploit available human judgements in the form of a hierarchical classification. Experimental results show that the retrieval using the generated queries is effective, particularly in terms of recall, while patent description is shown to be the most useful source for extracting query terms.
---
paper_title: Prior Art Retrieval Using Various Patent Document Fields Contents

paper_content:

In this paper, we report our approach to retrieve patent documents based on the prior art. We use the standard Information Retrieval (IR) techniques which contain indexing and retrieval processes. We use various combinations of document fields for the query formulation. Based on the evaluation summary, we achieve the best result for the combinations of invention-title, description and claims fields in terms of precision and recall.
---
paper_title: Transforming patents into prior-art queries

paper_content:

Searching for prior-art patents is an essential step for the patent examiner to validate or invalidate a patent application. In this paper, we consider the whole patent as the query, which reduces the burden on the user, and also makes many more potential search features available. We explore how to automatically transform the query patent into an effective search query, especially focusing on the effect of different patent fields. Experiments show that the background summary of a patent is the most useful source of terms for generating a query, even though most previous work used the patent claims.
---
paper_title: Applying the KISS Principle for the CLEF- IP 2010 Prior Art Candidate Patent Search Task

paper_content:

We present our experiments and results for the DCU CNGL participation in the CLEF-IP 2010 Candidate Patent Search Task. Our work applied standard information retrieval (IR) techniques to patent search. In addition, a very simple citation extraction method was applied to improve the results. This was our second consecutive participation in the CLEF-IP tasks. Our experiments in 2009 showed that many sophisticated approach to IR do not improve the retrieval effectiveness for this task. For this reason of we decided to apply only simple methods in 2010. These were demonstrated to be highly competitive with other participants. DCU submitted three runs for the Prior Art Candidate Search Task, two of these runs achieved the second and third ranks among the 25 runs submitted by nine different participants. Our best run achieved MAP of 0.203, recall of 0.618, and PRES of 0.523.
---
paper_title: A query model based on normalized log-likelihood

paper_content:

Leveraging information from relevance assessments has been proposed as an effective means for improving retrieval. We introduce a novel language modeling method which uses information from each assessed document and their aggregate. While most previous approaches focus either on features of the entire set or on features of the individual relevant documents, our model exploits features of both the documents and the set as a whole. When evaluated, we show that our model is able to significantly improve over state-of-art feedback methods.
---
paper_title: A study of query reformulation for patent prior art search with partial patent applications

paper_content:

Patents are used by legal entities to legally protect their inventions and represent a multi-billion dollar industry of licensing and litigation. In 2014, 326,033 patent applications were approved in the US alone -- a number that has doubled in the past 15 years and which makes prior art search a daunting, but necessary task in the patent application process. In this work, we seek to investigate the efficacy of prior art search strategies from the perspective of the inventor who wishes to assess the patentability of their ideas prior to writing a full application. While much of the literature inspired by the evaluation framework of the CLEF-IP competition has aimed to assist patent examiners in assessing prior art for complete patent applications, less of this work has focused on patent search with queries representing partial applications. In the (partial) patent search setting, a query is often much longer than in other standard IR tasks, e.g., the description section may contain hundreds or even thousands of words. While the length of such queries may suggest query reduction strategies to remove irrelevant terms, intentional obfuscation and general language used in patents suggests that it may help to expand queries with additionally relevant terms. To assess the trade-offs among all of these pre-application prior art search strategies, we comparatively evaluate a variety of partial application search and query reformulation methods. Among numerous findings, querying with a full description, perhaps in conjunction with generic (non-patent specific) query reduction methods, is recommended for best performance. However, we also find that querying with an abstract represents the best trade-off in terms of writing effort vs. retrieval efficacy (i.e., querying with the description sections only lead to marginal improvements) and that for such relatively short queries, generic query expansion methods help.
---
paper_title: Our Divided Patent System

paper_content:

In this comprehensive new study, we evaluate all substantive decisions rendered by any court in every patent case filed in 2008 and 2009 — decisions made between 2009 and 2013. We assess the outcome of litigation by technology and industry. We relate the outcomes of those cases to a host of variables, including variables related to the parties, the patents, and the courts in which those cases were litigated. We find dramatic differences in the outcomes of patent litigation by both technology and industry. For example, owners of patents in the pharmaceutical industry fare much better in dispositive litigation rulings than do owners of patents in the computer & electronics industry, and chemistry patents have much greater success in litigation than their software or biotech counterparts. Our results provide an important window into both patent litigation and the industry-specific battles over patent reform. And they suggest that the traditional narrative of industry-specific patent disputes, which pits the IT industries against the life sciences, is incomplete.
---
paper_title: A study on query expansion methods for patent retrieval

paper_content:

Patent retrieval is a recall-oriented search task where the objective is to find all possible relevant documents. Queries in patent retrieval are typically very long since they take the form of a patent claim or even a full patent application in the case of prior-art patent search. Nevertheless, there is generally a significant mismatch between the query and the relevant documents, often leading to low retrieval effectiveness. Some previous work has tried to address this mismatch through the application of query expansion (QE) techniques which have generally showed effectiveness for many other retrieval tasks. However, results of QE on patent search have been found to be very disappointing. We present a review of previous investigations of QE in patent retrieval, and explore some of these techniques on a prior-art patent search task. In addition, a novel method for QE using automatically generated synonyms set is presented. While previous QE techniques fail to improve over baseline retrieval, our new approach show statistically better retrieval precision over the baseline, although not for recall. In addition, it proves to be significantly more efficient than existing techniques. An extensive analysis to the results is presented which seeks to better understand situations where these QE techniques succeed or fail.
---
paper_title: Patent query reduction using pseudo relevance feedback

paper_content:

Queries in patent prior art search are full patent applications and much longer than standard ad hoc search and web search topics. Standard information retrieval (IR) techniques are not entirely effective for patent prior art search because of ambiguous terms in these massive queries. Reducing patent queries by extracting key terms has been shown to be ineffective mainly because it is not clear what the focus of the query is. An optimal query reduction algorithm must thus seek to retain the useful terms for retrieval favouring recall of relevant patents, but remove terms which impair IR effectiveness. We propose a new query reduction technique decomposing a patent application into constituent text segments and computing the Language Modeling (LM) similarities by calculating the probability of generating each segment from the top ranked documents. We reduce a patent query by removing the least similar segments from the query, hypothesising that removal of these segments can increase the precision of retrieval, while still retaining the useful context to achieve high recall. Experiments on the patent prior art search collection CLEF-IP 2010 show that the proposed method outperforms standard pseudo-relevance feedback (PRF) and a naive method of query reduction based on removal of unit frequency terms (UFTs).
---
paper_title: Leveraging conceptual lexicon: query disambiguation using proximity information for patent retrieval

paper_content:

Patent prior art search is a task in patent retrieval where the goal is to rank documents which describe prior art work related to a patent application. One of the main properties of patent retrieval is that the query topic is a full patent application and does not represent a focused information need. This query by document nature of patent retrieval introduces new challenges and requires new investigations specific to this problem. Researchers have addressed this problem by considering different information resources for query reduction and query disambiguation. However, previous work has not fully studied the effect of using proximity information and exploiting domain specific resources for performing query disambiguation. In this paper, we first reduce the query document by taking the first claim of the document itself. We then build a query-specific patent lexicon based on definitions of the International Patent Classification (IPC). We study how to expand queries by selecting expansion terms from the lexicon that are focused on the query topic. The key problem is how to capture whether an expansion term is focused on the query topic or not. We address this problem by exploiting proximity information. We assign high weights to expansion terms appearing closer to query terms based on the intuition that terms closer to query terms are more likely to be related to the query topic. Experimental results on two patent retrieval datasets show that the proposed method is effective and robust for query expansion, significantly outperforming the standard pseudo relevance feedback (PRF) and existing baselines in patent retrieval.
---
paper_title: The use of MMR, diversity-based reranking for reordering documents and producing summaries

paper_content:

This paper presents a method for combining ::: query-relevance with information-novelty in the context ::: of text retrieval and summarization. The Maximal ::: Marginal Relevance (MMR) criterion strives to reduce ::: redundancy while maintaining query relevance in ::: re-ranking retrieved documents and in selecting appropriate passages for text summarization. Preliminary results ::: indicate some benefits for MMR diversity ranking ::: in document retrieval and in single document summarization. ::: The latter are borne out by the recent results of the ::: SUMMAC conference in the evaluation of summarization ::: systems. However, the clearest advantage is demonstrated ::: in constructing non-redundant multi-document ::: summaries, where MMR results are clearly superior to ::: non-MMR passage selection.
---
paper_title: A study of query reformulation for patent prior art search with partial patent applications

paper_content:

Patents are used by legal entities to legally protect their inventions and represent a multi-billion dollar industry of licensing and litigation. In 2014, 326,033 patent applications were approved in the US alone -- a number that has doubled in the past 15 years and which makes prior art search a daunting, but necessary task in the patent application process. In this work, we seek to investigate the efficacy of prior art search strategies from the perspective of the inventor who wishes to assess the patentability of their ideas prior to writing a full application. While much of the literature inspired by the evaluation framework of the CLEF-IP competition has aimed to assist patent examiners in assessing prior art for complete patent applications, less of this work has focused on patent search with queries representing partial applications. In the (partial) patent search setting, a query is often much longer than in other standard IR tasks, e.g., the description section may contain hundreds or even thousands of words. While the length of such queries may suggest query reduction strategies to remove irrelevant terms, intentional obfuscation and general language used in patents suggests that it may help to expand queries with additionally relevant terms. To assess the trade-offs among all of these pre-application prior art search strategies, we comparatively evaluate a variety of partial application search and query reformulation methods. Among numerous findings, querying with a full description, perhaps in conjunction with generic (non-patent specific) query reduction methods, is recommended for best performance. However, we also find that querying with an abstract represents the best trade-off in terms of writing effort vs. retrieval efficacy (i.e., querying with the description sections only lead to marginal improvements) and that for such relatively short queries, generic query expansion methods help.
---
paper_title: Learning-Based pseudo-relevance feedback for patent retrieval

paper_content:

Pseudo-relevance feedback (PRF) is an effective approach in Information Retrieval but unfortunately many experiments have shown that PRF is ineffective in patent retrieval. This is because the quality of initial results in the patent retrieval is poor and therefore estimating a relevance model via PRF often hurts the retrieval performance due to off-topic terms. We propose a learning to rank framework for estimating the effectiveness of a patent document in terms of its performance in PRF. Specifically, the knowledge of effective feedback documents on past queries is used to estimate effective feedback documents for new queries. This is achieved by introducing features correlated with feedback document effectiveness. We use patent-specific contents to define such features. We then apply regression to predict document effectiveness given the proposed features. We evaluated the effectiveness of the proposed method on the patent prior art search collection CLEF-IP 2010. Our experimental results show significantly improved retrieval accuracy over a PRF baseline which expands the query using all top-ranked documents.
---
paper_title: Patent query reduction using pseudo relevance feedback

paper_content:

Queries in patent prior art search are full patent applications and much longer than standard ad hoc search and web search topics. Standard information retrieval (IR) techniques are not entirely effective for patent prior art search because of ambiguous terms in these massive queries. Reducing patent queries by extracting key terms has been shown to be ineffective mainly because it is not clear what the focus of the query is. An optimal query reduction algorithm must thus seek to retain the useful terms for retrieval favouring recall of relevant patents, but remove terms which impair IR effectiveness. We propose a new query reduction technique decomposing a patent application into constituent text segments and computing the Language Modeling (LM) similarities by calculating the probability of generating each segment from the top ranked documents. We reduce a patent query by removing the least similar segments from the query, hypothesising that removal of these segments can increase the precision of retrieval, while still retaining the useful context to achieve high recall. Experiments on the patent prior art search collection CLEF-IP 2010 show that the proposed method outperforms standard pseudo-relevance feedback (PRF) and a naive method of query reduction based on removal of unit frequency terms (UFTs).
---
paper_title: On Term Selection Techniques for Patent Prior Art Search

paper_content:

In this paper, we investigate the influence of term selection on retrieval performance on the CLEF-IP prior art test collection, using the Description section of the patent query with Language Model (LM) and BM25 scoring functions. We find that an oracular relevance feedback system that extracts terms from the judged relevant documents far outperforms the baseline and performs twice as well on MAP as the best competitor in CLEF-IP 2010. We find a very clear term selection value threshold for use when choosing terms. We also noticed that most of the useful feedback terms are actually present in the original query and hypothesized that the baseline system could be substantially improved by removing negative query terms. We tried four simple automated approaches to identify negative terms for query reduction but we were unable to notably improve on the baseline performance with any of them. However, we show that a simple, minimal interactive relevance feedback approach where terms are selected from only the first retrieved relevant document outperforms the best result from CLEF-IP 2010 suggesting the promise of interactive methods for term selection in patent prior art search.
---
paper_title: Improving retrievability of patents in prior-art search

paper_content:

Prior-art search is an important task in patent retrieval. The success of this task relies upon the selection of relevant search queries. Typically terms for prior-art queries are extracted from the claim fields of query patents. However, due to the complex technical structure of patents, and presence of terms mismatch and vague terms, selecting relevant terms for queries is a difficult task. During evaluating the patents retrievability coverage of prior-art queries generated from query patents, a large bias toward a subset of the collection is experienced. A large number of patents either have a very low retrievability score or can not be discovered via any query. To increase the retrievability of patents, in this paper we expand prior-art queries generated from query patents using query expansion with pseudo relevance feedback. Missing terms from query patents are discovered from feedback patents, and better patents for relevance feedback are identified using a novel approach for checking their similarity with query patents. We specifically focus on how to automatically select better terms from query patents based on their proximity distribution with prior-art queries that are used as features for computing similarity. Our results show, that the coverage of prior-art queries can be increased significantly by incorporating relevant queries terms using query expansion.
---
paper_title: A study on query expansion methods for patent retrieval

paper_content:

Patent retrieval is a recall-oriented search task where the objective is to find all possible relevant documents. Queries in patent retrieval are typically very long since they take the form of a patent claim or even a full patent application in the case of prior-art patent search. Nevertheless, there is generally a significant mismatch between the query and the relevant documents, often leading to low retrieval effectiveness. Some previous work has tried to address this mismatch through the application of query expansion (QE) techniques which have generally showed effectiveness for many other retrieval tasks. However, results of QE on patent search have been found to be very disappointing. We present a review of previous investigations of QE in patent retrieval, and explore some of these techniques on a prior-art patent search task. In addition, a novel method for QE using automatically generated synonyms set is presented. While previous QE techniques fail to improve over baseline retrieval, our new approach show statistically better retrieval precision over the baseline, although not for recall. In addition, it proves to be significantly more efficient than existing techniques. An extensive analysis to the results is presented which seeks to better understand situations where these QE techniques succeed or fail.
---
paper_title: Wikipedia-based query phrase expansion in patent class search

paper_content:

Relevance feedback methods generally suffer from topic drift caused by word ambiguities and synonymous uses of words. Topic drift is an important issue in patent information retrieval as people tend to use different expressions describing similar concepts causing low precision and recall at the same time. Furthermore, failing to retrieve relevant patents to an application during the examination process may cause legal problems caused by granting an existing invention. A possible cause of topic drift is utilizing a relevance feedback-based search method. As a way to alleviate the inherent problem, we propose a novel query phrase expansion approach utilizing semantic annotations in Wikipedia pages, trying to enrich queries with phrases disambiguating the original query words. The idea was implemented for patent search where patents are classified into a hierarchy of categories, and the analyses of the experimental results showed not only the positive roles of phrases and words in retrieving additional relevant documents through query expansion but also their contributions to alleviating the query drift problem. More specifically, our query expansion method was compared against relevance-based language model, a state-of-the-art query expansion method, to show its superiority in terms of MAP on all levels of the classification hierarchy.
---
paper_title: Patent Query Formulation by Synthesizing Multiple Sources of Relevance Evidence

paper_content:

Patent prior art search is a task in patent retrieval with the goal of finding documents which describe prior art work related to a query patent. A query patent is a full patent application composed of hundreds of terms which does not represent a single focused information need. Fortunately, other relevance evidence sources (i.e., classification tags and bibliographical data) provide additional details about the underlying information need. In this article, we propose a unified framework that integrates multiple relevance evidence components for query formulation. We first build a query model from the textual fields of a query patent. To overcome the term mismatch, we expand this initial query model with the term distribution of documents in the citation graph, modeling old and recent domain terminology. We build an IPC lexicon and perform query expansion using this lexicon incorporating proximity information. We performed an empirical evaluation on two patent datasets. Our results show that employing the temporal features of documents has a precision enhancing effect, while query expansion using IPC lexicon improves the recall of the final rank list.
---
paper_title: Recommending patents based on latent topics

paper_content:

The availability of large volumes of granted patents and applications, all publicly available on the Web, enables the use of sophisticated text mining and information retrieval methods to facilitate access and analysis of patents. In this paper we investigate techniques to automatically recommend patents given a query patent. This task is critical for a variety of patent-related analysis problems such as finding relevant citations, research of relevant prior art, and infringement analysis. We investigate the use of latent Dirichlet allocation and Dirichlet multinomial regression to represent patent documents and to compute similarity scores. We compare our methods with state-of-the-art document representations and retrieval techniques and demonstrate the effectiveness of our approach on a collection of US patent publications.
---
paper_title: An effective approach to document retrieval via utilizing WordNet and recognizing phrases

paper_content:

Noun phrases in queries are identified and classified into four types: proper names, dictionary phrases, simple phrases and complex phrases. A document has a phrase if all content words in the phrase are within a window of a certain size. The window sizes for different types of phrases are different and are determined using a decision tree. Phrases are more important than individual terms. Consequently, documents in response to a query are ranked with matching phrases given a higher priority. We utilize WordNet to disambiguate word senses of query terms. Whenever the sense of a query term is determined, its synonyms, hyponyms, words from its definition and its compound words are considered for possible additions to the query. Experimental results show that our approach yields between 23% and 31% improvements over the best-known results on the TREC 9, 10 and 12 collections for short (title only) queries, without using Web data.
---
paper_title: Predicting query performance

paper_content:

We develop a method for predicting query performance by computing the relative entropy between a query language model and the corresponding collection language model. The resulting clarity score measures the coherence of the language usage in documents whose models are likely to generate the query. We suggest that clarity scores measure the ambiguity of a query with respect to a collection of documents and show that they correlate positively with average precision in a variety of TREC test sets. Thus, the clarity score may be used to identify ineffective queries, on average, without relevance information. We develop an algorithm for automatically setting the clarity score threshold between predicted poorly-performing queries and acceptable queries and validate it using TREC data. In particular, we compare the automatic thresholds to optimum thresholds and also check how frequently results as good are achieved in sampling experiments that randomly assign queries to the two classes.
---
paper_title: PatNet: A Lexical Database for the Patent Domain

paper_content:

In the patent domain Boolean retrieval is particularly common. But despite the importance of Boolean retrieval, there is not much work in current research assisting patent experts in formulating such queries. Currently, these approaches are mostly limited to the usage of standard dictionaries, such as WordNet, to provide synonymous expansion terms. In this paper we present a new approach to support patent searchers in the query generation process. We extract a lexical database, which we call PatNet, from real query sessions of patent examiners of the United Patent and Trademark Office (USPTO). PatNet provides several types of synonym relations. Further, we apply several query term expansion strategies to improve the precision measures of PatNet in suggesting expansion terms. Experiments based on real query sessions of patent examiners show a drastic increase in precision, when considering support of the synonym relations, US patent classes, and word senses.
---
paper_title: Adaptive relevance feedback in information retrieval

paper_content:

Relevance Feedback has proven very effective for improving retrieval accuracy. A difficult yet important problem in all relevance feedback methods is how to optimally balance the original query and feedback information. In the current feedback methods, the balance parameter is usually set to a fixed value across all the queries and collections. However, due to the difference in queries and feedback documents, this balance parameter should be optimized for each query and each set of feedback documents. In this paper, we present a learning approach to adaptively predict the optimal balance coefficient for each query and each collection. We propose three heuristics to characterize the balance between query and feedback information. Taking these three heuristics as a road map, we explore a number of features and combine them using a regression approach to predict the balance coefficient. Our experiments show that the proposed adaptive relevance feedback is more robust and effective than the regular fixed-coefficient feedback.
---
paper_title: Selecting good expansion terms for pseudo-relevance feedback

paper_content:

Pseudo-relevance feedback assumes that most frequent terms in the pseudo-feedback documents are useful for the retrieval. In this study, we re-examine this assumption and show that it does not hold in reality - many expansion terms identified in traditional approaches are indeed unrelated to the query and harmful to the retrieval. We also show that good expansion terms cannot be distinguished from bad ones merely on their distributions in the feedback documents and in the whole collection. We then propose to integrate a term classification process to predict the usefulness of expansion terms. Multiple additional features can be integrated in this process. Our experiments on three TREC collections show that retrieval effectiveness can be much improved when term classification is used. In addition, we also demonstrate that good terms should be identified directly according to their possible impact on the retrieval effectiveness, i.e. using supervised learning, instead of unsupervised learning.
---
paper_title: Mining Query Logs of USPTO Patent Examiners

paper_content:

In this paper we analyze a highly professional search setting of patent examiners of the United Patent and Trademark Office USPTO. We gain insight into the search behavior of USPTO patent examiners to explore ways for enhancing query generation in patent searching. We show that query generation is highly patent domain specific and patent examiners follow a strict scheme for generating text queries. Means to enhance query generation in patent search are to suggest synonyms and equivalents, co-occurring terms and keyword phrases to the searchable features of the invention. Further, we show that term networks including synonyms and equivalents can be learned from the query logs for automatic query expansion in patent searching.
---
paper_title: Effect of Log-Based Query Term Expansion on Retrieval Effectiveness in Patent Searching

paper_content:

In this paper we study the impact of query term expansion QTE using synonyms on patent document retrieval. We use an automatically generated lexical database from USPTO query logs, called PatNet, which provides synonyms and equivalents for a query term. Our experiments on the CLEF-IP 2010 benchmark dataset show that automatic query expansion using PatNet tends to decrease or only slightly improve the retrieval effectiveness, with no significant improvement. An analysis of the retrieval results shows that PatNet does not have generally a negative effect on the retrieval effectiveness. Recall is drastically improved for query topics, where the baseline queries achieve, on average, only low recall values. But we have not detected any commonality that allows us to characterize these queries. So we recommend using PatNet for semi-automatic QTE in Boolean retrieval, where expanding query terms with synonyms and equivalents with the aim of expanding the query scope is a common practice.
---
paper_title: Acquiring Lexical Knowledge from Query Logs for Query Expansion in Patent Searching

paper_content:

Query expansion is a crucial step in recall-oriented domains such as Patent Searching. Currently, automatic query expansion in patent search is mostly based on statistical measures. Additional query terms are extracted from the query documents based on entropy measures. To automate query expansion in patent searching, we acquire lexical knowledge from Query Logs of USPTO Patent Examiners. Results show good performance in query expansion and patent searching using the lexical database. This will help improving (semi-) automated query expansion in patent searching.
---
paper_title: Analyzing query logs of USPTO examiners to identify useful query terms in patent documents for query expansion in patent searching: a preliminary study

paper_content:

In an attempt to improve retrieval systems for the patent domain, significant efforts are invested to assist researchers in formulating better queries, preferably via automated query generation. Current work on query generation in patent retrieval is mostly based on statistical measures without considering whether these terms are the best choice. To learn from actual queries being posed by experts, we analyze query logs from USPTO patent examiners. Results show that US examiners pick the majority of query terms from the claim section, a large fraction of which, in turn, coincide with the subject feature terms which determine the extent of the protection of the patent right. Considering the lessons learned from evaluating existing search logs will help in improving (semi-) automated query generation.
---
paper_title: Using query logs of USPTO patent examiners for automatic query expansion in patent searching

paper_content:

In the patent domain significant efforts are invested to assist researchers in formulating better queries, preferably via automated query expansion. Currently, automatic query expansion in patent search is mostly limited to computing co-occurring terms for the searchable features of the invention. Additional query terms are extracted automatically from patent documents based on entropy measures. Learning synonyms in the patent domain for automatic query expansion has been a difficult task. No dedicated sources providing synonyms for the patent domain, such as patent domain specific lexica or thesauri, are available. In this paper we focus on the highly professional search setting of patent examiners. In particular, we use query logs to learn synonyms for the patent domain. For automatic query expansion, we create term networks based on the query logs specifically for several USPTO patent classes. Experiments show good performance in automatic query expansion using these automatically generated term networks. Specifically, with a larger number of query logs for a specific patent US class available the performance of the learned term networks increases.
---
paper_title: Multiple Retrieval Models and Regression Models for Prior Art Search

paper_content:

This paper presents the system called PATATRAS (PATent and Article Tracking, Retrieval and AnalysiS) realized at the Humboldt University for the IP track of CLEF 2009. Our approach presents three main characteristics: 1. The usage of multiple retrieval models (KL, Okapi) and term index definitions (lemma, phrase, concept) for the three languages considered in the present track (English, French, German) producing ten different sets of ranked results. 2. The merging of the different results based on multiple regression models using an additional validation set created from the patent collection. 3. The exploitation of patent metadata and of the citation structures for creating restricted initial working sets of patents and for producing a final re-ranking regression model. As we exploit specific metadata of the patent documents and the citation relations only at the creation of initial working sets and during the final post ranking step, our architecture remains generic and easy to extend.
---
paper_title: Applying the KISS Principle for the CLEF- IP 2010 Prior Art Candidate Patent Search Task

paper_content:

We present our experiments and results for the DCU CNGL participation in the CLEF-IP 2010 Candidate Patent Search Task. Our work applied standard information retrieval (IR) techniques to patent search. In addition, a very simple citation extraction method was applied to improve the results. This was our second consecutive participation in the CLEF-IP tasks. Our experiments in 2009 showed that many sophisticated approach to IR do not improve the retrieval effectiveness for this task. For this reason of we decided to apply only simple methods in 2010. These were demonstrated to be highly competitive with other participants. DCU submitted three runs for the Prior Art Candidate Search Task, two of these runs achieved the second and third ranks among the 25 runs submitted by nine different participants. Our best run achieved MAP of 0.203, recall of 0.618, and PRES of 0.523.
---
paper_title: Experiments with Citation Mining and Key-Term Extraction for Prior Art Search

paper_content:

This technical note presents the system built for the IP track of CLEF 2010 based on PATATRAS (PATent and Article Tracking, Retrieval and AnalysiS), the modular search infrastructure initially realized for CLEF IP 2009. We largely reused the system of the previous CLEF IP but at a relatively smaller scale and with the improvement of three main components: • A new citation mining tool based on Conditional Random Fields (CRF). • A key-term extraction module developed for technical and scientific documents and adapted to patent document structures using a vast ranges of metrics, features and a bagged decision tree. • An improvement of our multi-domain terminological database called GRISP. We used the Okapi BM25 and the Indri retrieval models for the prior art task and a KNN model for the automatic classification task under the IPC subclasses. In both tasks, specific final re-ranking techniques were used, including multiple regression models based on SVM. Although the Prior Art task was more challenging and we used a more limited number of retrieval models, we maintained similar results as last year. We performed, however, miserably at the classification task, and we consider that an instance-based KNN algorithm is not competitive with standard classifiers based on preliminary large scale training.
---
paper_title: Patent Query Formulation by Synthesizing Multiple Sources of Relevance Evidence

paper_content:

Patent prior art search is a task in patent retrieval with the goal of finding documents which describe prior art work related to a query patent. A query patent is a full patent application composed of hundreds of terms which does not represent a single focused information need. Fortunately, other relevance evidence sources (i.e., classification tags and bibliographical data) provide additional details about the underlying information need. In this article, we propose a unified framework that integrates multiple relevance evidence components for query formulation. We first build a query model from the textual fields of a query patent. To overcome the term mismatch, we expand this initial query model with the term distribution of documents in the citation graph, modeling old and recent domain terminology. We build an IPC lexicon and perform query expansion using this lexicon incorporating proximity information. We performed an empirical evaluation on two patent datasets. Our results show that employing the temporal features of documents has a precision enhancing effect, while query expansion using IPC lexicon improves the recall of the final rank list.
---
paper_title: An IPC-based vector space model for patent retrieval

paper_content:

Determining requirements when searching for and retrieving relevant information suited to a user's needs has become increasingly important and difficult, partly due to the explosive growth of electronic documents. The vector space model (VSM) is a popular method in retrieval procedures. However, the weakness in traditional VSM is that the indexing vocabulary changes whenever changes occur in the document set, or the indexing vocabulary selection algorithms, or parameters of the algorithms, or if wording evolution occurs. The major objective of this research is to design a method to solve the afore-mentioned problems for patent retrieval. The proposed method utilizes the special characteristics of the patent documents, the International Patent Classification (IPC) codes, to generate the indexing vocabulary for presenting all the patent documents. The advantage of the generated indexing vocabulary is that it remains unchanged, even if the document sets, selection algorithms, and parameters are changed, or if wording evolution occurs. Comparison of the proposed method with two traditional methods (entropy and chi-square) in manual and automatic evaluations is presented to verify the feasibility and validity. The results also indicate that the IPC-based indexing vocabulary selection method achieves a higher accuracy and is more satisfactory.
---
paper_title: On the role of classification in patent invalidity searches

paper_content:

Searches on patents to determine prior art violations are often cumbersome and require extensive manpower to accomplish successfully. When time is constrained, an automatically generated list of candidate patents may decrease search costs and improve search efficiency. We examine whether semantic relations inferred from the pseudo-hierarchy of patent classifications can contribute to the recognition of related patents. We examine a similarity measure for hierarchically-ordered patent classes and subclasses and return a ranked list of candidate patents, using a similarity measure that has demonstrated its effectiveness when applied to WordNet ontologies. We then demonstrate that this ranked list of candidate patents allows us to better constrain the effort needed to examine for prior art violations on a target patent.
---
paper_title: Comparison of IPC and USPC classification systems in patent prior art searches

paper_content:

Patent classification systems are used to help scrutinize patent applications for possible violations of the novelty and non-obviousness/inventive steps of a patentability test. There are several different patent classification systems in use today, each with a different underlying philosophy and approach. We compare the two most widely-used patent classification systems -- the IPC and USPC -- and examine their ability to help re-rank patents based on similarity. We observed a significant improvement in MAP, Recall@100, and nDCG when using these systems to re-rank our retrieved document set, demonstrating their overall utility in patent searches.
---
paper_title: Cluster-based patent retrieval using international patent classification system

paper_content:

A patent collection provides a great test-bed for cluster-based information retrieval. International Patent Classification (IPC) system provides a hierarchical taxonomy with 5 levels of specificity. We regard IPC codes of patent applications as cluster information, manually assigned by patent officers according to their subjects. Such manual cluster provides advantages over auto-matically built clusters using document term similarities. There are previous researches that successfully apply cluster-based retrieval models using language modeling. We develop cluster-based language models that employ advantages of having manually clustered documents.
---
paper_title: The effect of citation analysis on query expansion for patent retrieval

paper_content:

Patent prior art search is a type of search in the patent domain where documents are searched for that describe the work previously carried out related to a patent application. The goal of this search is to check whether the idea in the patent application is novel. Vocabulary mismatch is one of the main problems of patent retrieval which results in low retrievability of similar documents for a given patent application. In this paper we show how the term distribution of the cited documents in an initially retrieved ranked list can be used to address the vocabulary mismatch. We propose a method for query modeling estimation which utilizes the citation links in a pseudo relevance feedback set. We first build a topic dependent citation graph, starting from the initially retrieved set of feedback documents and utilizing citation links of feedback documents to expand the set. We identify the important documents in the topic dependent citation graph using a citation analysis measure. We then use the term distribution of the documents in the citation graph to estimate a query model by identifying the distinguishing terms and their respective weights. We then use these terms to expand our original query. We use CLEF-IP 2011 collection to evaluate the effectiveness of our query modeling approach for prior art search. We also study the influence of different parameters on the performance of the proposed method. The experimental results demonstrate that the proposed approach significantly improves the recall over a state-of-the-art baseline which uses the link-based structure of the citation graph but not the term distribution of the cited documents.
---
paper_title: Simple pre and post processing strategies for patent searching in CLEF intellectual property track 2009

paper_content:

The objective of the 2009 CLEF-IP Track was to find documents that constitute prior art for a given patent. We explored a wide range of simple preprocessing and post-processing strategies, using Mean Average Precision (MAP) for evaluation purposes. Once determined the best document representation, we tuned a classical Information Retrieval engine in order to perform the retrieval step. Finally, we explored two different post-processing strategies. In our experiments, using the complete IPC codes for filtering purposes led to greater improvements than using 4-digits IPC codes. The second postprocessing strategy was to exploit the citations of retrieved patents in order to boost scores of cited patents. Combining all selected strategies, we computed optimal runs that reached a MAP of 0.122 for the training set, and a MAP of 0.129 for the official 2009 CLEF-IP XL set.
---
paper_title: Building queries for prior-art search

paper_content:

Prior-art search is a critical step in the examination procedure of a patent application. This study explores automatic query generation from patent documents to facilitate the time-consuming and labor-intensive search for relevant patents. It is essential for this task to identify discriminative terms in different fields of a query patent, which enables us to distinguish relevant patents from non-relevant patents. To this end we investigate the distribution of terms occurring in different fields of the query patent and compare the distributions with the rest of the collection using language modeling estimation techniques. We experiment with term weighting based on the Kullback-Leibler divergence between the query patent and the collection and also with parsimonious language model estimation. Both of these techniques promote words that are common in the query patent and are rare in the collection. We also incorporate the classification assigned to patent documents into our model, to exploit available human judgements in the form of a hierarchical classification. Experimental results show that the retrieval using the generated queries is effective, particularly in terms of recall, while patent description is shown to be the most useful source for extracting query terms.
---
paper_title: Query-Driven Mining of Citation Networks for Patent Citation Retrieval and Recommendation

paper_content:

Prior art search or recommending citations for a patent application is a challenging task. Many approaches have been proposed and shown to be useful for prior art search. However, most of these methods do not consider the network structure for integrating and diffusion of different kinds of information present among tied patents in the citation network. In this paper, we propose a method based on a time-aware random walk on a weighted network of patent citations, the weights of which are characterized by contextual similarity relations between two nodes on the network. The goal of the random walker is to find influential documents in the citation network of a query patent, which can serve as candidates for drawing query terms and bigrams for query refinement. The experimental results on CLEF-IP datasets (CLEF-IP 2010 and CLEF-IP 2011) show the effectiveness of encoding contextual similarities (common classification codes, common inventor, and common applicant) between nodes in the citation network. Our proposed approach can achieve significantly better results in terms of recall and Mean Average Precision rates compared to strong baselines of prior art search.
---
paper_title: Multiple Retrieval Models and Regression Models for Prior Art Search

paper_content:

This paper presents the system called PATATRAS (PATent and Article Tracking, Retrieval and AnalysiS) realized at the Humboldt University for the IP track of CLEF 2009. Our approach presents three main characteristics: 1. The usage of multiple retrieval models (KL, Okapi) and term index definitions (lemma, phrase, concept) for the three languages considered in the present track (English, French, German) producing ten different sets of ranked results. 2. The merging of the different results based on multiple regression models using an additional validation set created from the patent collection. 3. The exploitation of patent metadata and of the citation structures for creating restricted initial working sets of patents and for producing a final re-ranking regression model. As we exploit specific metadata of the patent documents and the citation relations only at the creation of initial working sets and during the final post ranking step, our architecture remains generic and easy to extend.
---
paper_title: Multilayer source selection as a tool for supporting patent search and classification

paper_content:

In this paper we present a method that can be used to attain specific objectives in a typical prior art search process. The objectives are first to assist patent searchers in understanding the underlying technical concepts of a patent by identifying relevant international patent classification (IPC) codes and second to help them conduct a filtered search based on automatically selected IPCs. We view the automated selection of IPCs as a collection selection problem from the domain of distributed information retrieval (DIR) that can be addressed using existing DIR methods, which we extend and adapt for the patent domain. Our work exploits the intellectually assigned classifications codes that are used to categorize patents and to facilitate patent searches. In our method, manually assigned IPC codes of patent documents are used to cluster, distribute and index patents through hundreds or thousands of sub-collections. We propose a new multilayer collection selection method that effectively suggests classification codes exploiting the hierarchical classification schemes such as IPC/CPC. The new method in addition to utilizing the topical relevance of IPCs at a particular level of interest exploits the topical relevance of their ancestors in the IPC hierarchy and aggregates those multiple estimations of relevance to a single estimation. Experimental results on the CLEF-IP 2011 dataset show that the proposed approach outperforms state-of-art methods from the DIR domain not only in identifying relevant IPC codes but also in retrieving relevant patent documents given a patent query.
---
paper_title: Simple vs. sophisticated approaches for patent prior-art search

paper_content:

Patent prior-art search is concerned with finding all filed patents relevant to a given patent application. We report a comparison between two search approaches representing the state-of-the-art in patent prior-art search. The first approach uses simple and straightforward information retrieval (IR) techniques, while the second uses much more sophisticated techniques which try to model the steps taken by a patent examiner in patent search. Experiments show that the retrieval effectiveness using both techniques is statistically indistinguishable when patent applications contain some initial citations. However, the advanced search technique is statistically better when no initial citations are provided. Our findings suggest that less time and effort can be exerted by applying simple IR approaches when initial citations are provided.
---
paper_title: Automated Patent Categorization and Guided Patent Search using IPC as Inspired by MeSH and PubMed

paper_content:

Document search on PubMed, the pre-eminent database for biomedical literature, relies on the annotation of its documents with relevant terms from the Medical Subject Headings ontology (MeSH) for improving recall through query expansion. Patent documents are another important information source, though they are considerably less accessible. One option to expand patent search beyond pure keywords is the inclusion of classification information: Since every patent is assigned at least one class code, it should be possible for these assignments to be automatically used in a similar way as the MeSH annotations in PubMed. In order to develop a system for this task, it is necessary to have a good understanding of the properties of both classification systems. This report describes our comparative analysis of MeSH and the main patent classification system, the International Patent Classification (IPC). We investigate the hierarchical structures as well as the properties of the terms/classes respectively, and we compare the assignment of IPC codes to patents with the annotation of PubMed documents with MeSH terms. Our analysis shows a strong structural similarity of the hierarchies, but significant differences of terms and annotations. The low number of IPC class assignments and the lack of occurrences of class labels in patent texts imply that current patent search is severely limited. To overcome these limits, we evaluate a method for the automated assignment of additional classes to patent documents, and we propose a system for guided patent search based on the use of class co-occurrence information and external resources.
---
paper_title: Patent search using IPC classification vectors

paper_content:

Finding similar patents is a challenging task in patent information retrieval. A patent application is often a starting point to find similar inventions. Keyword search for similar patents requires significant domain expertise and may not fetch relevant results. We propose a novel representation for patents and use a two stage approach to find similar patents. Each patent is represented as an IPC class vector. Citation network of patents is used to propagate these vectors from a node (patent) to its neighbors (cited patents). Thus, each patent is represented as a weighted combination of its IPC information as well as of its neighbors. A query patent is represented as a vector using its IPC information and similar patents can be simply found by comparing this vector with vectors of patents in the corpus. Text based search is used to re-rank this solution set to improve precision. We experiment with two similarity measures and re-ranking strategies to empirically show that our representation is effective in improving both precision and recall of queries of CLEF-2011 dataset.
---
paper_title: Exploratory Professional Search through Semantic Post-Analysis of Search Results

paper_content:

Professional Search is usually a recall-oriented problem. For helping the user to get efficiently a concise overview, to quickly restrict the search space and to make sense of the results, in this article we present an exploratory strategy for professional search that is based on semantic post-analysis of the classical search results (of keyword based queries). The described strategy can exploit the metadata that are already available, as well as the results of textual clustering and entity mining that can be performed at query time. The outcome of this process (i.e. metadata, clusters and entities grouped in categories) complement the ranked list of results produced from the core search engine with useful information for the user. This extra information is useful not only for providing a concise overview of the search results, but also for supporting a faceted and session-based interaction scheme that allows the users to restrict their focus gradually and to explore other related information. To tackle the corresponding configuration requirements of this process, we show how one can exploit the (constantly evolving) Linked Data for specifying the entities of interest and for providing further information about the identified entities. In this article, apart from detailing the steps of this process, we present applications of this approach in the marine domain and in the domain of patent search.
---
paper_title: An Evaluation of an Interactive Federated Patent Search System

paper_content:

Patent search tasks are challenging and often require many hours or even days to be completed. Patent search systems that integrate multiple search tools could assist patent examiners to complete the demanding patent search tasks by using the set of search tools most suitable for the task at hand. PerFedPat is an interactive patent search system designed on this principle and based on the federated search approach and the ezDL framework. PerFedPat provides core services to search multiple online patent resources, while hiding complexity from the end user. The extensible architecture of the system enables the integration and parallel use of multiple search tools. In this paper, we present part of a user study of the PerFedPat system and we also discuss the results, mostly focused on the research question: could the patent examiners efficiently use a patent search system that integratesmultiple resources, search tools and UIs?
---
paper_title: On Term Selection Techniques for Patent Prior Art Search

paper_content:

In this paper, we investigate the influence of term selection on retrieval performance on the CLEF-IP prior art test collection, using the Description section of the patent query with Language Model (LM) and BM25 scoring functions. We find that an oracular relevance feedback system that extracts terms from the judged relevant documents far outperforms the baseline and performs twice as well on MAP as the best competitor in CLEF-IP 2010. We find a very clear term selection value threshold for use when choosing terms. We also noticed that most of the useful feedback terms are actually present in the original query and hypothesized that the baseline system could be substantially improved by removing negative query terms. We tried four simple automated approaches to identify negative terms for query reduction but we were unable to notably improve on the baseline performance with any of them. However, we show that a simple, minimal interactive relevance feedback approach where terms are selected from only the first retrieved relevant document outperforms the best result from CLEF-IP 2010 suggesting the promise of interactive methods for term selection in patent prior art search.
---
paper_title: A visual semantic framework for innovation analytics

paper_content:

In this demo we present a semantic framework for innovation and patent analytics powered by Mined Semantic Analysis (MSA). Our framework provides cognitive assistance to its users through a Web-based visual and interactive interface. First, we describe building a conceptual knowledge graph by mining user-generated encyclopedic textual corpus for semantic associations. Then, we demonstrate applying the acquired knowledge to support many cognition and knowledge based use cases for innovation analysis including technology exploration and landscaping, competitive analysis, literature and prior art search and others.
---
paper_title: PerFedPat: An integrated federated system for patent search

paper_content:

Abstract We present PerFedPat, an interactive patent search system based on the federated search approach and the ezDL framework. PerFedPat provides core services to search, using a federated method, multiple online patent resources (currently Espacenet, Google patents, Patentscope and the MAREC collection), thus providing parallel access to multiple patent sources. PerFedPat hides complexity from the end user who uses a common single query tool for querying all patent datasets at the same time. PerFedPat provides cores services such as Boolean and fielded search, merging, grouping and filtering of results, and offers support for query history and search sessions. The second innovative feature of PerFedPat is that it has a pluggable and extensible architecture and it enables the use of multiple search tools which are integrated in PerFedPat. Currently tools are integrated for IPC classification search, faceted navigation, clustering of search results and machine translation of queries. As a result the system is able to provide a rich, personalized information seeking experience for different types of patent searches, potentially exploiting techniques from diverse areas such as distributed information retrieval, semantic search, machine learning and human–computer interaction. The system is available for download from the internet.
---
paper_title: Evaluation of machine-learning protocols for technology-assisted review in electronic discovery

paper_content:

Abstract Using a novel evaluation toolkit that simulates a human reviewer in the loop, we compare the effectiveness of three machine-learning protocols for technology-assisted review as used in document review for discovery in legal proceedings. Our comparison addresses a central question in the deployment of technology-assisted review: Should training documents be selected at random, or should they be selected using one or more non-random methods, such as keyword search or active learning? On eight review tasks -- four derived from the TREC 2009 Legal Track and four derived from actual legal matters -- recall was measured as a function of human review effort. The results show that entirely non-random training methods, in which the initial training documents are selected using a simple keyword search, and subsequent training documents are selected by active learning, require substantially and significantly less human review effort (P
---
paper_title: Measuring Semantic Relatedness using Mined Semantic Analysis

paper_content:

Mined Semantic Analysis (MSA) is a novel distributional semantics approach which employs data mining techniques. MSA embraces knowledge-driven analysis of natural languages. It uncovers implicit relations between concepts by mining for their associations in target encyclopedic corpora. MSA exploits not only target corpus content but also its knowledge graph (e.g., "See also" link graph of Wikipedia). Empirical results show competitive performance of MSA compared to prior state-of-the-art methods for measuring semantic relatedness on benchmark data sets. Additionally, we introduce the first analytical study to examine statistical significance of results reported by different semantic relatedness methods. Our study shows that, top performing results could be statistically equivalent though mathematically different. The study positions MSA as one of state-of-the-art methods for measuring semantic relatedness.
---
paper_title: The Market Value of Blocking Patent Citations

paper_content:

There is a growing literature that aims at assessing the private value of knowledge assets and patents. It has been shown that patents and their quality as measured by citations received by future patents contribute significantly to the market value of firms beyond their R&D stocks. This paper goes one step further and distinguishes between different types of forward citations patents can receive at the European Patent Office. While a patent can be cited as non-infringing state of the art, it can also be cited because it threatens the novelty of patent applications ('blocking citations'). Empirical results from a market value model for a sample of large, R&D-intensive U.S., European and Japanese firms show that patents frequently cited as blocking references have a higher economic value for their owners than patents cited for nonblocking reasons. This finding adds to the patent value literature by showing that different types of patent citations carry different information on the economic value of patents. The result further suggests that the total number of forward citations can be an imprecise measure of patent value.
---
paper_title: A New Look at Patent Quality: Relating Patent Prosecution to Validity

paper_content:

The paper uses two hand-collected datasets to implement a novel research design for analyzing the precursors to patent quality. Operationalizing patent "quality" as legal validity, the paper analyzes the relation between Federal Circuit decisions on patent validity and three sets of data about the patents: quantitative features of the patents themselves, textual analysis of the patent documents, and data collected from the prosecution histories of the patents. The paper finds large and statistically significant relations between ex post validity and both textual features of the patents and ex ante aspects of the prosecution history (especially prior art submissions and the existence of internal patent office appeals before issuance). The results demonstrate the importance of refocusing analysis of patent quality on replicable indicators like validity, and the value that more comprehensive collection of prosecution history data can have for improving the output of the patent prosecution process.
---
paper_title: A Penny for Your Quotes : Patent Citations and the Value of Innovations

paper_content:

The use of patents in economic research has been seriously hindered by the fact that patents vary enormously in their importance or value, and hence, simple patent counts cannot be informative about innovative output. The purpose of this article is to put forward patent counts weighted by citations as indicators of the value of innovations, thereby overcoming the limitations of simple counts. The empirical analysis of a particular innovation (Computed Tomography scanners) indeed shows a close association between citation-based patent indices and independent measures of the social value of innovations in that field. Moreover, the weighting scheme appears to be nonlinear (increasing) in the number of citations, implying that the informational content of citations rises at the margin. As in previous studies, simple patent counts are found to be highly correlated with contemporaneous RD however, here the association is within a field over time rather than cross-sectional.
---
paper_title: SIMPLE: A Strategic Information Mining Platform for Licensing and Execution

paper_content:

Intellectual Properties (IP), such as patents and trademarks, are one of the most critical assets in today’s enterprises and research organizations. They represent the core innovation and differentiators of an organization. When leveraged effectively, they not only protect a business from its competition, but also generate significant opportunities in licensing, execution, long term research and innovation. In certain industries, e. g., Pharmaceutical industry, patents lead to multi-billion dollar revenue per year. In this paper, we present a holistic information mining solution, called SIMPLE, which mines large corpus of patents and scientific literature for insights. Unlike much prior work that deals with specific aspects of analytics, SIMPLE is an integrated and end-to-end IP analytics solution which addresses a wide range of challenges in patent analytics such as the data complexity, scale, and nomenclature issues. It encompasses techniques for patent data processing and modeling, analytics algorithms, web interface and web services for analytics service delivery and end-user interaction. We use real-world case studies to demonstrate the effectiveness of SIMPLE.
---
paper_title: Finding nuggets in IP portfolios: core patent mining through textual temporal analysis

paper_content:

Patents are critical for a company to protect its core technologies. Effective patent mining in massive patent databases can provide companies with valuable insights to develop strategies for IP management and marketing. In this paper, we study a novel patent mining problem of automatically discovering core patents (i.e., patents with high novelty and influence in a domain). We address the unique patent vocabulary usage problem, which is not considered in traditional word-based statistical methods, and propose a topic-based temporal mining approach to quantify a patent's novelty and influence. Comprehensive experimental results on real-world patent portfolios show the effectiveness of our method.
---
paper_title: Patent Maintenance Recommendation with Patent Information Network Model

paper_content:

Patents are of crucial importance for businesses, because they provide legal protection for the invented techniques, processes or products. A patent can be held for up to 20 years. However, large maintenance fees need to be paid to keep it enforceable. If the patent is deemed not valuable, the owner may decide to abandon it by stopping paying the maintenance fees to reduce the cost. For large companies or organizations, making such decisions is difficult because too many patents need to be investigated. In this paper, we introduce the new patent mining problem of automatic patent maintenance prediction, and propose a systematic solution to analyze patents for recommending patent maintenance decision. We model the patents as a heterogeneous time-evolving information network and propose new patent features to build model for a ranked prediction on whether to maintain or abandon a patent. In addition, a network-based refinement approach is proposed to further improve the performance. We have conducted experiments on the large scale United States Patent and Trademark Office (USPTO) database which contains over four million granted patents. The results show that our technique can achieve high performance.
---
paper_title: Modeling Patent Quality: A System for Large-scale Patentability Analysis using Text Mining

paper_content:

Current patent systems face a serious problem of declining quality of patents as the larger number of ap- plications make it difficult for patent officers to spend enough time for evaluating each application. For building a better patent system, it is necessary to define a public consensus on the quality of patent applications in a quantitative way. In this article, we tackle the problem of assessing the quality of patent applications based on machine learning and text mining techniques. For each patent application, our tool automatically computes a score called patentability, which indicates how likely it is that the application will be approved by the patent office. We employ a new statis- tical prediction model to estimate examination results (approval or rejection) based on a large data set including 0.3 million patent applications. The model computes the patentability score based on a set of feature variables including the text contents of the specification documents. Experimental results showed that our model outperforms a conven- tional method which uses only the structural properties of the documents. Since users can access the estimated result through a Web-browser-based GUI, this system allows both patent examiners and applicants to quickly detect weak applications and to find their specific flaws.
---
paper_title: COA: finding novel patents through text analysis

paper_content:

In recent years, the number of patents filed by the business enterprises in the technology industry are growing rapidly, thus providing unprecedented opportunities for knowledge discovery in patent data. One important task in this regard is to employ data mining techniques to rank patents in terms of their potential to earn money through licensing. Availability of such ranking can substantially reduce enterprise IP (Intellectual Property) management costs. Unfortunately, the existing software systems in the IP domain do not address this task directly. Through our research, we build a patent ranking software, named COA (Claim Originality Analysis) that rates a patent based on its value by measuring the recency and the impact of the important phrases that appear in the "claims" section of a patent. Experiments show that COA produces meaningful ranking when comparing it with other indirect patent evaluation metrics--citation count, patent status, and attorney's rating. In reallife settings, this tool was used by beta-testers in the IBM IP department. Lawyers found it very useful in patent rating, specifically, in highlighting potentially valuable patents in a patent cluster. In this article, we describe the ranking techniques and system architecture of COA. We also present the results that validate its effectiveness.
---
paper_title: Citation Frequency and the Value of Patented Inventions

paper_content:

Through a survey, economic value estimates were obtained on 962 inventions made in the United States and Germany and on which German patent renewal fees were paid to full-term expiration in 1995. A search of subsequent U.S. and German patents yielded a count of citations to those patents. Patents renewed to full term were significantly more valuable than patents allowed to expire before their full term. The higher an invention's economic value estimate was, the more the relevant patent was subsequently cited.
---
paper_title: Exploring Legal Patent Citations for Patent Valuation

paper_content:

Effective patent valuation is important for patent holders. Forward patent citations, widely used in assessing patent value, have been considered as reflecting knowledge flows, just like paper citations. However, patent citations also carry legal implication, which is important for patent valuation. We argue that patent citations can either be technological citations that indicate knowledge transfer or be legal citations that delimit the legal scope of citing patents. In this paper, we first develop citation-network based methods to infer patent quality measures at either the legal or technological dimension. Then we propose a probabilistic mixture approach to incorporate both the legal and technological dimensions in patent citations, and an iterative learning process that integrates a temporal decay function on legal citations, a probabilistic citation network based algorithm and a prediction model for patent valuation. We learn all the parameters together and use them for patent valuation. We demonstrate the effectiveness of our approach by using patent maintenance status as an indicator of patent value and discuss the insights we learned from this study.
---
paper_title: Latent graphical models for quantifying and predicting patent quality

paper_content:

The number of patents filed each year has increased dramatically in recent years, raising concerns that patents of questionable validity are restricting the issuance of truly innovative patents. For this reason, there is a strong demand to develop an objective model to quantify patent quality and characterize the attributes that lead to higher-quality patents. In this paper, we develop a latent graphical model to infer patent quality from related measurements. In addition, we extract advanced lexical features via natural language processing techniques to capture the quality measures such as clarity of claims, originality, and importance of cited prior art. We demonstrate the effectiveness of our approach by validating its predictions with previous court decisions of litigated patents.
---
paper_title: How to Count Patents and Value Intellectual Property: The Uses of Patent Renewal and Application Data

paper_content:

Patent counts are very imperfect measures of innovative output. This paper discusses how additional data-the number of years a patent is renewed and the number of countries in which protection for the same invention is sought - can be used to improve on counts in studies which require a measure of the extent of innovation. A simple renewal based weighting scheme is proposed which may remove half of the noise in patent counts as a measure of innovative output. The paper also illustrates how these data can be used to estimate the value of the proprietary rights created by the patent laws. The parameters estimated in this analysis can be used to answer a series of questions related to the value of patents. We illustrate with estimates of how the value of patent protection would vary under alternative legal rules and renewal fees, and with estimates of the international flows of returns from the patent system. Recent progress in the development of databases has increased the potential for this type of analysis.
---
paper_title: Market value and patent citations

paper_content:

We explore the usefulness of patent citations as a measure of the "importance" of a firm's patents, as indicated by the stock market valuation of the firm's intangible stock of knowledge. Using patents and citations for 1963--1995, we estimate Tobin's q equations on the ratios of R&D to assets stocks, patents to R&D, and citations to patents. We find that each ratio significantly affects market value, with an extra citation per patent boosting market value by 3%. Further findings indicate that "unpredictable" citations have a stronger effect than the predictable portion, and that self-citations are more valuable than external citations.
---
paper_title: Our Divided Patent System

paper_content:

In this comprehensive new study, we evaluate all substantive decisions rendered by any court in every patent case filed in 2008 and 2009 — decisions made between 2009 and 2013. We assess the outcome of litigation by technology and industry. We relate the outcomes of those cases to a host of variables, including variables related to the parties, the patents, and the courts in which those cases were litigated. We find dramatic differences in the outcomes of patent litigation by both technology and industry. For example, owners of patents in the pharmaceutical industry fare much better in dispositive litigation rulings than do owners of patents in the computer & electronics industry, and chemistry patents have much greater success in litigation than their software or biotech counterparts. Our results provide an important window into both patent litigation and the industry-specific battles over patent reform. And they suggest that the traditional narrative of industry-specific patent disputes, which pits the IT industries against the life sciences, is incomplete.
---
paper_title: Using Data Analytics Tools to Supplement Traditional Research and Analysis in Forecasting Case Outcomes

paper_content:

The legal profession appears to be on the brink of a major transformation, as new data analytics tools promise to revolutionize the ways lawyers practice law. One important aspect of this transformation is case forecasting -- that is, the way lawyers evaluate potential case outcomes. When a lawyer decides whether to initiate a new lawsuit, for example, or whether to settle an existing lawsuit, the lawyer must try to accurately assess the likely outcome of the case were it to proceed in litigation. This article examines the future role of data analytics in case forecasting, including its interplay with the method of predictive analysis lawyers have traditionally employed. This analysis has important implications not only for the practice of law, but also for legal pedagogy, which will need to conform to these expected changes in legal practice.
---
paper_title: The Law Machine

paper_content:

In a low-rise building in Menlo Park, Calif., just upstairs from a Mexican restaurant and a nail salon, a Stanford University spin-off is crunching data in ways that could shake the foundations of the legal profession.Here, a small group of patent lawyers and computer scientists is applying the latest in machine learning and natural-language processing to reams of documents related to intellectual property lawsuits. The result is a massive statistical database on IP litigation like nothing the world has seen before.Which attorney has the best track record in defending against semiconductor-related infringement claims? Has a particular judge ruled on cases involving patent trolls, and if so, what was the outcome? Which companies tend to go to trial, and which settle out of court? By offering up such information, the database provides corporate lawyers, law lirms, and government agencies with hard numbers that will reduce the guesswork, as well as the enormous expense, of patent litigation. In short, the company is building a “law machine,” from whichcomes its name: Lex Machina.
---
paper_title: Understanding the Realities of Modern Patent Litigation

paper_content:

Sixteen years ago, two of us published the first detailed empirical look at patent litigation. In this Article, we update and expand the earlier study with a new hand-coded data set. We evaluate all substantive decisions rendered by any court in every patent case filed in 2008 and 2009 — decisions made between 2009 and 2013. We consider not just patent validity but also infringement and unenforceability. Moreover, we relate the outcomes of those cases to a host of variables, including variables related to the parties, the patents, and the courts in which those cases were litigated. The result is a comprehensive picture of the outcomes of modern patent litigation, one that confirms conventional wisdom in some respects but upends it in others. In particular, we find a surprising amount of continuity in the basic outcomes of patent lawsuits over the past twenty years, despite rather dramatic changes in who brought patent suits during that time.
---
paper_title: SIMPLE: A Strategic Information Mining Platform for Licensing and Execution

paper_content:

Intellectual Properties (IP), such as patents and trademarks, are one of the most critical assets in today’s enterprises and research organizations. They represent the core innovation and differentiators of an organization. When leveraged effectively, they not only protect a business from its competition, but also generate significant opportunities in licensing, execution, long term research and innovation. In certain industries, e. g., Pharmaceutical industry, patents lead to multi-billion dollar revenue per year. In this paper, we present a holistic information mining solution, called SIMPLE, which mines large corpus of patents and scientific literature for insights. Unlike much prior work that deals with specific aspects of analytics, SIMPLE is an integrated and end-to-end IP analytics solution which addresses a wide range of challenges in patent analytics such as the data complexity, scale, and nomenclature issues. It encompasses techniques for patent data processing and modeling, analytics algorithms, web interface and web services for analytics service delivery and end-user interaction. We use real-world case studies to demonstrate the effectiveness of SIMPLE.
---
paper_title: Exploratory analytics on patent data sets using the SIMPLE platform

paper_content:

Abstract Exploratory Analytics is the process of analyzing data for the purpose of forming hypotheses. Patent data sets, because they are relatively large and diverse and because they consist of a mixture of structured and unstructured information present a formidable challenge and a great opportunity in applying exploratory analytics techniques. In this paper we describe methods we have implemented for effective exploratory analytics on patent data sets using an interactive approach and a web based software tool called SIMPLE. We use real-world case studies to demonstrate the effectiveness of our exploratory analytics approach in the discovery of useful information from the patent corpus.
---
paper_title: SIMPLE: Interactive Analytics on Patent Data

paper_content:

Intellectual Properties (IP), such as patents and trademarks, are one of the most critical assets in today’s enterprises and research organizations. They represent the core innovation and differentiators of an organization. When leveraged effectively, they not only protect freedom of action, but also generate significant opportunities in licensing, execution, long term research and innovation. In this paper, we expand upon a previous paper describing a solution called SIMPLE, which mines large corpus of patents and scientific literature for insights. In this paper we focus on the interactive analytics aspects of SIMPLE, which allow the analyst to explore large unstructured information collections containing mixed information in a dynamic way. We use real-world case studies to demonstrate the effectiveness of interactive analytics in SIMPLE.
---

Title: Patent Retrieval: A Literature Review

Section 1: Introduction
Description 1: Write about the significance of patents, why patent retrieval is critical, and the unique challenges faced in this field.

Section 2: Patent documents and kind codes
Description 2: Describe the structure of patent documents, the various elements within them, and the importance of kind codes.

Section 3: Patent classification
Description 3: Explain how patents are classified using different systems like IPC and CPC, and the relevance of these classifications.

Section 4: Patent families
Description 4: Define what patent families are and discuss their relevance in the context of patent retrieval and prior art search.

Section 5: Data and evaluation tracks
Description 5: Provide an overview of various evaluation tracks and data collections used in patent data analysis, focusing on their role in benchmarking patent retrieval systems.

Section 6: Patent retrieval tasks
Description 6: Discuss the different tasks within patent retrieval, such as prior art search, patentability search, infringement search, and more.

Section 7: Patent retrieval methods
Description 7: Present a comprehensive review of different methods in patent retrieval, categorized by their approach (e.g., keyword-based, PRF, semantic-based, metadata-based, interactive).

Section 8: Related topics
Description 8: Lightly touch on related topics such as patent quality assessment, litigation, and technology licensing, highlighting key challenges and opportunities.

Section 9: Concluding remarks
Description 9: Summarize the key points covered in the review, discuss the current state of the art, and suggest future directions for research in patent retrieval.