Unit Information: SIT772 Database and Information Retrieval
Trimester: 2020 T1
Assessment 2: Information Retrieval Techniques Problem Solving Task This document supplies the detailed information on assessment tasks for this unit.
Key information
• Due: Week 12-Sunday, 7 June 2020, 23:59 (AEST)
• Weighting: 30%
• Submit: Through CloudDeakin
Learning Outcomes
This assessment assesses the following Unit Learning Outcomes (ULO) and related Graduate Learning Outcomes (GLO):
Purpose
This task evaluates the student’s technical skills in the management of unstructured data, with potential usage in real applications. This assessment supports student understandings of the techniques related to unstructured data management and data processing
Instructions and Submission Guide
This is an individual assessment task. Students are required to submit ONE written report. • Read these instructions and the following questions.
• ONE written report with the name as using student ID_givenname_A2.pdf, e.g., 123456_Kevin_A2.pdf
• The report must be submitted via CloudDeakin assessment portal. The wrong submission venue or the wrong submitted file may lead to the penalty.
Question 1 (Index Construction): [8 Marks]
Suppose you have joined a search engine development team to design a search algorithm based on both the Vector model and the Boolean model.
You have collected the following documents (unstructured) and plan to apply an index technique to convert them into an inverted index.
Doc 1：data science is field to use scientific method, process, algorithm, system to extract knowledge.
Doc 2：data mining is the process to discover pattern in large data to involve method at the database system.
Doc 3：information system is the study of network of hardware and software that people use
to process data.
To answer the below questions, you have to provide the detailed procedures step by step. You need to remove all stop words and punctuation before the process of creating the inverted index. After that, please complete the following steps:
Question 1.1: [2 Marks]
Create a merged inverted list including the within-document frequencies for each term.
Question 1.2: [2 Marks]
Use the index created as above to create a dictionary and the related posting file.
Question 1.3: [2 Marks]
Please design three Boolean queries, (for example, web AND search) and list the relevant documents for each query. Each query must contain at least two keywords while no one keyword appears in one document only.
Question 1.4: [2 Marks]
Please use the Vector model to query on the inverted index, and compare the result with the Boolean model. (Hint: you can use cosine similarity and set a similarity threshold).
Question 2 (IR Evaluation): [15 Marks]
In this question, you are required to evaluate the performance of different search engines. • First, please select two of the three search engines you are familiar, https://www.google.com.au/, https://www.bing.com/?cc=au, https://au.yahoo.com/.
Figure 1: Select the search engine located in Australia
• Second, you are required to constrain your query to www.reuters.com by specifying: e.g., Given a keyword query “high-tech global”, you need to write into the search box with “high-tech global site:www.reuters.com” at your selected search engine website. Below is the example provided for you to use google, shown in Figure 2 and Figure 3.
[Penalty is applied if your results do not follow the instruction.]
Figure 2: Search “high-tech global” within the website www.reuters.com
Figure 3: Search Results of “high-tech global” within the website www.reuters.com
• Third, please choose one target from the following list, and design two queries to search in both search engines. So both query 1 and query 2 have to be tested in both search engines.
✔ Target 1: Online tutors have more incomes in worldwide education marketing. ✔ Target 2: Online education expanding, awaits innovation.
✔ Target 3: Innovations in online learning for regional people.
• Finally, select the first 20 results in both search engines, if they return the target news, then mark them as relevant documents, otherwise, they are irrelevant. Note: assume there are 12 relevant documents in total (retrieved and not-retrieved). If you search more than 12 relevant documents, you can simply regard some to be irrelevant for practice.
The following questions are based on your search results.
Question 2.1: [3 Marks]
List your target, results and designed search queries (You can use any keywords you think are related to the target news, even if the keywords are not contained the news text). For each result, you can click the link and go to the page, and write down a summary with 2~3 sentences for each result if you think this result is relevant. At your report, you are required to provide the URL address and the summary to explain why they are relevant to the queries.
Question 2.2: [3 Marks]
Get the precision and recall values for 20 documents for query 1 in search engine 1. Interpolate them to 11 standard recall levels. Then plot them into a chart. Get the precision and recall values for 20 documents for query 1 in search engine 2. Interpolate them to 11 standard recall levels. Then plot them into the same chart.
Question 2.3: [3 Marks]
Get the precision and recall values for 20 documents for query 2 in search engine 1. Interpolate them to 11 standard recall levels. Then plot them into a chart. Get the precision and recall values for 20 documents for query 2 in search engine 2. Interpolate them to 11 standard recall levels. Then plot them into the same chart.
Question 2.4: [3 Marks]
Now find the average interpolated precision of query 1 and query 2 for search engine 1 and plot it into a chart. So you will have total of 3 interpolated curves in one single chart. Now find the average interpolated precision of query 1 and query 2 for search engine 2 and plot it into the same chart. So, you will have total of 3 interpolated curves in one single chart.
Question 2.5: [3 Marks]
Plot the average interpolated values for Search Engine 1 and Search Engine 2 on one single chart, and compare the algorithms in terms of precision and recall. Which search engine do you think is superior? Why?
Question 3 (Innovation Concept Design of Decentralized Web Search Engine): [7 Marks] In this question, you are required to make the great brainstorming for concept design. As we
learnt, all the current web search engine companies host the web data in their own web server. The data generators cannot control their own data. In the near future, it might be highly desirable for the worldwide researchers or companies to design a new type of web search engine, denoted as decentralized or distributed web search engine. Lots of start-ups also make investment in such area. The benefit is to achieve data privacy protection, data use transparency, and removing the centralization of data management.
[https://en.wikipedia.org/wiki/Distributed_search_engine,
https://www.quora.com/Do-you-know-any-decentralized-search-engine,
https://gadget.co.za/web-gets-decentralised/].
Example: Decentralized Internet Project.
https://coins.newbium.com/post/16998-decentralized-internet-project-cardstack
In this challenging task, your design should include several important aspects:
Question 3.1: [2 Marks]
How to maintain the data using index? Must have 200~300 words to describe the design.
Question 3.2: [2 Mark]
Question 3.2. How to answer keyword queries in the new types of data environment? Must have 200~300 words to describe the design.
Question 3.3: [1 Mark]
Question 3.3. How to evaluate such new web search engine system? Must have 200~300 words to describe the design.
Question 3.4: [2 Marks]
Question 3.4. Provide the system structure concept design diagram. Must have a diagram and 200~300 words to explain the diagram.
Assessment feedback
General feedback to the class will be provided via CloudDeakin-Discussion Forum. The formal assessment feedback will be released with the marks in CloudDeakin altogether.
Extension requests
Requests for extensions should be made to Unit/Campus Chairs 3 days early before the assessment due date. Unit Chair: Jianxin Li, Jianxin.li@deakin.edu.au
Special consideration
You may be eligible for special consideration if circumstances beyond your control prevent you from undertaking or completing an assessment task at the scheduled time.
See the following link for advice on the application process: http://www.deakin.edu.au/students/studying/assessment-and-results/special-consideration
Assessment feedback
Detailed written feedback will be provided within two weeks of submission.
Referencing
You must correctly use Harvard referencing in this assessment. See the Deakin referencing guide.
Academic integrity, plagiarism and collusion
Plagiarism and collusion constitute extremely serious breaches of academic integrity. They are forms of cheating, and severe penalties are associated with them, including cancellation of marks for a specific assignment, for a specific unit or even exclusion from the course. If you are ever in doubt about how to properly use and cite a source of information refer to the referencing site above.
Plagiarism occurs when a student passes off as the student’s own work, or copies without acknowledgement as to its authorship, the work of any other person or resubmits their own work from a previous assessment task.
Collusion occurs when a student obtains the agreement of another person for a fraudulent purpose, with the intent of obtaining an advantage in submitting an assignment or other work.
Work submitted may be reproduced and/or communicated by the university for the purpose of assuring academic integrity of submissions: https://www.deakin.edu.au/students/study support/referencing/academic-integrity
