COMP4037 Research Methods Coursework 3
Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit
COMP4037 Research Methods
Coursework 3: (Abridged) Literature Review
Number of Credits: 50% of module
Recommended hours: 40-50 hours (for distinction level)
Re-Assessment Deadline: 21 August by 16:00
Late submissions will incur a penalty of 10% per day including weekends and bank holidays. Submit a PDF file to Moodle-a link to a submission link is provided a few days before the deadline.
The aim of this coursework is:
● For you to search for and examine a number of academic papers in your area of interest, summarize them, compare them, categorize them. You may explain the methods used and how they have gathered and used evidence to support their findings.
This supports the following Learning Objectives on the module:
● Ability to critically appraise and choose research papers and methods, and justify their application to appropriate research problems
● Identify and critically discuss managerial, political, economic and ethical problems associated with the development and execution of a research project
● Good understanding of research process and the application of appropriate research approaches to given problems
● Enhanced skills to write research proposals to implement projects
● The ability to evaluate available appropriate approaches to a given situation
● Problem solving, ability to compare and contrast based on understanding and experience of research skills
Task Description and Context
Questions: If you have any questions about the coursework, please ask them 1) in-class, 2) after class, or 3) in the chat of the Research Methods Teams module. Individual emails to me will result in me asking you to use one of these routes due to the large numbers of students.
Choose a topic: You will choose an area in computer science related to your dissertation topic. Choose 10 research papers, and then write a (maximum) 2000-word literature review discussing the research papers. The word limit does not include the list of references at the end. Note that this coursework specification itself is already 1,800 words. Summarize each research paper using the same template provided in coursework 1. Then categorize them. You may examine the methods used in the papers, and include this in your paper summaries.
Finding Research Papers
The area of research that you choose is ideally the same as for your dissertation topic. Arrange a meeting with your supervisor and ask them for an appropriate literature review topic. You may ask them for some research papers to start with and include those in this coursework. Also look for a recent survey paper on your chosen topic. Include this in your literature review. Include a total of 10 research papers that are in that area. This includes any research papers that your supervisor recommends to you. Each research paper in your review should have a DOI. Review the advice in the session on Finding Relevant Research from earlier in the module to help you search the research literature and find appropriate papers.
If you do not yet have a supervisor because you are a two-year enhanced MSc student, then start with a recent survey paper on the topic you have chosen. You can also look for highly-cited papers on your topic and recent publications. You can discuss your topic with Bob to see if it’s appropriate.
Extra Help on Finding Research Papers
See the following tutorials for help on finding research papers.
(Laramee 2023) Robert S Laramee, Research Methods: FInding Relevant Research Papers
Research Methods: Finding Relevant Research Papers
(Laramee 2023) Robert S Laramee, Research Methods: Feedback on Finding Relevant Research Papers Workshop
Research Methods: Feedback on Finding Relevant Research Papers Workshop
Summarizing Research Papers
For each research paper you:
● Summarize each paper using the same strategy and template as in Coursework 1. Each summary is 1-2 paragraphs. Remember you cannot exceed the 2,000 word limit. The 2,000 word limit does not include references. See Firat et al (2022) for a good example of what your paper summaries can look like.
● Don’t forget the type of evaluation. The following are some examples of types of evaluation:
○ Qualitative or quantitative methods
○ Comparison with previous algorithms based on performance time, accuracy, or size of input data
○ Conducting a controlled user study
○ Case study: testing a specific hypothesis or demonstrating a given use-case scenario/application
○ Interviews
○ Questionnaires
○ Focus groups
○ Mathematical proofs
○ Studying a collection of documents as in a survey or literature review
Explain how the researchers found evidence to evaluate their work and back up their findings. Did they test the accuracy/speed/performance/biases of their new algorithms on specific datasets? Do they record measurements of hardware performance? Examine themes from interviews or focus groups?
Extra Help on Summarizing Research Papers
See Laramee (2011) and Firat et al (2022) for an example of this including research paper summaries that are 1-2 paragraphs each. See also the extra help and references section in Coursework 1 for more help on this.
(Laramee 2016) Robert S Laramee, How To Read a Scientific Research Paper, There are four examples with two minute research paper summaries in this lecture.
How To Read a Scientific Research Paper: Extracting the Essentials
(Laramee 2018) Robert S Laramee, How to Deal with Hundreds or Thousands of Research Papers, There are four examples with two minute research paper summaries in this lecture.
How to Deal with Hundreds or Thousands of Research Papers
(Laramee 2023) Robert S Laramee, Extracting the Essentials of a Computer Science Research Paper,
Research Methods: Extracting the Essentials of a Computer Science Research Paper
Categorization
Then you categorize the papers. This means putting the papers into different groups based on something they have in common. This could be based on many different factors such as: recurring themes, type of user, application domain, size of data processed (input or output), performance, the type of evaluation, the type of data processed (input or output), a commonality in sub-topic. Then, you provide a brief summary, explaining how you categorized the papers and the different dimensions you chose to do so. Provide either a table or figure that illustrates how your papers are grouped and organized.
Extra Help on Categorization
For a detailed discussion of ways to categorize research papers, see Section 3.2. Of McNabb and Laramee (2019), “Developing a Literature Classification (How To)”. You may also see several examples of this in the Survey of Surveys (SoS) paper by McNabb and Laramee (2017).
Liam McNabb and Robert S Laramee, Survey of Surveys (SoS)-Mapping the Landscape of Survey Papers in Information Visualization, Talk at the Eurovis 2017 Conference, Barcelona, Spain, June 2017
Robert S Laramee, How to Write a Visualization Survey Paper, Practice Talk for the Eurographics 2019 Conference
How To Write a Visualization Survey Paper: Practice Talk for Eurographics 2019
Literature Browser
Add the papers in your literature review to an online literature browser–Survis (Beck et al 2016). You can set up your Survis literature browser web page on GitHub or Gitlab. Take a screenshot of your SurVis web page and include it in your report. Make sure you put the URL to your SurVis page in the caption of your screenshot. For instructions on how to set up SurVis with a number of examples, please visit:
https://github.com/fabian-beck/survis
Figure: A high-quality SurVis web page set up by Dylan Rees: http://visbooks.swansea.ac.uk/ (Rees and Laramee 2019)
Supplementary Figures
Include a histogram of years on the x-axis and number of papers on the y-axis. You may use the histogram provided by the SurVis web page if you like. Include this figure in your report. See the histogram in Firat et al (2022) for an example of this. Also include one interesting or representative image from each paper you summarize and include it in the literature review with a descriptive caption and a citation. See Firat et al (2022) for a good example of this.
Figure: An example of a histogram from Firat et al (2022).
Extra Help on Supplementary Figures
See Table 2 of McNabb and Laramee (2019) for an extensive discussion of candidate supplementary figures that can be included in a literature review.
Robert S Laramee, How to Write a Visualization Survey Paper, Practice Talk for the Eurographics 2019 Conference in Genoa, Italy, 6-10 May 2019.
How To Write a Visualization Survey Paper: Practice Talk for Eurographics 2019
List of References
Finally, you include a reference list including references for each research paper and any other papers or resources that are used, quoted, or relevant. Make sure your references are complete, just like the ones below used in this coursework. This aspect will be assessed.
Literature Review Template
Title of Literature Review: Including Topic
Name, Student Number
The following is the outline of how your literature review is organized:
1. Introduction and Motivation
Approximately 1-2 paragraphs. In the introduction you describe what the topic is and why it is important. See McNabb and Larameee (2019) for more details about how to write a good introduction. The introduction and motivation section includes a meta-data figure(s). See Table 2 of McNabb and Laramee (2019) for a detailed description of meta-data and meta-data figures. For example, see Figure 1 above.
1.1. Field Challenges
Approximately 1 paragraph possibly including a bulleted or numbered list. What are the common challenges in the topic you have chosen? These are general challenges that every researcher faces when investigating the given research topic. See page 2, Section 1.3. Of Edmunds et al for an example (Edmunds et al, 2012).
1.2. Survey Scope
Approximately 1-2 paragraphs. Describe what topics are included and which topics are not (the subject matter). See Section 1.5. of McNabb and Laramee (2019) for a detailed description of the Survey Scope.
1.3. Search Methodology
Approximately 1-2 paragraphs. Describe how you searched for related research papers here. Which supervisor did you contact? Which research paper did they suggest you start with? Did your supervisor provide you with any keywords to search with? Did you find a recent survey paper on your topic? Include a screenshot from ConnectedPapers.com with a descriptive caption stating which paper is highlighted in the screenshot.
1.4. Classification of Literature and Organization
Approximately 1-2 paragraphs. Here you describe your literature classification and provide an overview. Your classification could include a summary table. You can see many examples of this in McNabb and Laramee (2017). This section will include a screenshot of your Survis web page with the supporting URL.
2. Paper Summaries
Use 1 paragraph for each paper summary. See Firat et al (2022) for an example. Optional: to facilitate the paper summary reading, you may include indicators such as (paper 1) in the text. This style is not normally used, however, it will make the assessment easier and clearer for the reader. Whatever you do, do not include the paper title, authors, and words from the paper summary template, e.g., concept, implementation etc, in your descriptions. Paper details such as title, authors, etc appear in the Bibliography. Paper summaries appear as standard natural language text, just like this paragraph. See Firat et al (2022) page 4, column 2, paragraph 1, for an example. See also Wang and Laramee (2022) page 10, column 1, for more nice examples of paper summaries with images. You are also allowed to use tables in your literature review.
For summarizing survey papers include 1) a short introduction to the topic (what) 2), why it is important, and 3) a description of the paper categorization/classification.
References
Include your list of references here. Include the DOIs of your papers. You can use the same style as in this coursework. They do not count in the total word count. References are listed in alphabetical order by first author surname just like they are in this assignment.
Extra Help on Courseworks 2 & 3
● Supplementary Questions and Answers:Research Methods: Questions and Answers on Courseworks 2 & 3
● More Questions and Answers on Courseworks 2 & 3
● https://youtu.be/6DJYgh8B1uE
Academic Integrity
This is an individual assessment that should consist of your own work. Throughout the paper, remember that any direct quotations from the papers are in quotation marks and an in-text citation added to the reference list. You are permitted to use any material (e.g., diagrams and quotations) but you must make it very clear what your sources are. The University provides detailed advice on academic integrity:
https://www.nottingham.ac.uk/studyingeffectively/referencing/integrity/index.aspx
The university has also developed a policy on AI and ChatGPT. Please review it here:
Welcome - ChatGPT Guidance for Students
Please review this guidance. Submissions that demonstrate a lack of that integrity will be treated under appropriate disciplinary procedure.
References and Extra Help
(Beck et al, 2016) F. Beck, S. Koch and D. Weiskopf, Visual Analysis and Dissemination of Scientific Literature Collections with SurVis, in IEEE Transactions on Visualization and Computer Graphics, vol. 22, no. 1, pp. 180-189, 31 Jan. 2016, doi: 10.1109/TVCG.2015.2467757.
(Edmunds et al, 2012) Matthew Edmunds, Robert S. Laramee, Guoning Chen, Nelson Max, Eugene Zhang, and Colin Ware, Surface Based Flow Visualization, Computers & Graphics, Vol. 36, No. 8, pages 974-990 (December 2012) ( PDF file, http://dx.doi.org/10.1016/j.cag.2012.07.006 )
(Firat et al 2022) Elif E Firat, Alark Joshi, and Robert S Laramee, Interactive Visualization Literacy: The State-of-the-Art, Information Visualization, Volume 21, Number 3, pages 285–310, 2022 ( PDF file, https://doi.org/10.1177/14738716221081831 )
(Laramee 2011) Robert S. Laramee, How to Read a Visualization Research Paper: Extracting the Essentials, IEEE Computer Graphics and Applications (IEEE CG&A), Vol. 31, No. 3, May/June 2011, pages 78-82 ( PDF file, web page, http://dx.doi.org/10.1109/MCG.2011.44 )
(McNabb and Laramee, 2017) Liam McNabb and Robert S. Laramee, Survey of Surveys (SoS)-Mapping the Landscape of Survey Papers in Information Visualization in Computer Graphics Forum (CGF) , Volume 36, Number 3, (June) 2017, pages 589-617, ( PDF , Supplementary PDF , http://sos.swansea.ac.uk , http://dx.doi.org/doi:10.1111/cgf.13212, EuroVis 2017 talk )
(McNabb and Larameee, 2019) Liam McNabb and Robert S. Laramee, How to Write a Visualization Survey Paper: A Starting Point, in EUROGRAPHICS 2019, Education Papers, pages 29-39, 6-10 May 2019, Genova, Italy ( PDF file, http://dx.doi.org/10.2312/eged.20191026 )
(Rees and Laramee, 2019) Dylan Rees and Robert S. Laramee, A Survey of Information Visualization Books, Computer Graphics Forum (CGF), Volume 38, Issue 1, February 2019, pages 610-646 ( PDF file , http://visbooks.swansea.ac.uk , https://doi.org/10.1111/cgf.13595 )
(Wang and Laramee 2022) Qiru Wang and Robert S Laramee, EHR STAR: The State-of-the-Art in Interactive EHR Visualization, Computer Graphics Forum, Volume 41, Number 1, pages 69-105, February 2022 ( PDF file, slides, https://doi.org/10.1111/cgf.14424 )
Assessment Criteria
We will assess your report on the following criteria:
● The quality of the language and writing
● The description of your categorization and how well the literature review is organized
● The quality, accuracy, and clarity of the paper summaries
● The inclusion and quality of at least one meta-data figure
● The inclusion and quality of a supporting SurVis web page
● The completeness and accuracy of your literature references
Grade 80-100. An exceptional literature review of the research is provided, similar to that of a good PhD student. The quality of the review is at a publishable level. Every aspect of the literature review template is covered and completed at a publishable level.
● The introduction and motivation are written at a publishable level along with a meta-data figure.
● The field challenges are correctly identified and described at a publishable level.
● The scope is described at a publishable level.
● The search methodology is correctly implemented and described at a publishable level along with a screenshot from ConnectedPapers.com.
● The quality of the literature classification and organization is done well and presented at publishable level including a screenshot + functioning URL of the Survis web page.
● The paper summaries are written and presented at a publishable level.
● The references are complete, in alphabetical order by first-author surname, and a DOI is provided for each paper.
Grade 70-79. A very clear literature review of the research is provided, similar to that of a beginning PhD student. The quality of the review is very good. Every aspect of the literature review template is covered and completed at a very good level.
● The introduction and motivation are written at a very good level along with a meta-data figure.
● The field challenges are correctly identified and described at a very good level.
● The scope is described at a very good level.
● The search methodology is correctly implemented and described at a very good level along with a screenshot from ConnectedPapers.com.
● The quality of the literature classification and organization is done well and presented at a very good level including a screenshot + functioning URL of the Survis web page.
● The paper summaries are written and presented at a very good level.
● The references are complete, in alphabetical order by first-author surname, and a DOI is provided for each paper.
Grade 60-69. A good literature review of the research is provided and most sections are clear. Every aspect of the literature review template is covered and completed though perhaps falling short of a thorough and professional level with perhaps some incomplete or mixed up aspects of the template.
● The introduction and motivation are written along with a meta-data figure but some aspects could be clearer.
● The field challenges are correctly identified and described but could be presented more clearly.
● The scope is described correctly but could be improved.
● The search methodology is correctly implemented and described along with a screenshot from ConnectedPapers.com but not necessarily to a high standard.
● The quality of the literature classification and organization is done well and presented at including a screenshot + functioning URL of the Survis web page but not necessarily to a high standard.
● The paper summaries are written and presented but could be improved.
● The references are complete, in alphabetical order by first-author surname, and a DOI is provided for each paper.
Grade 50-59. A literature review of the research is provided and most sections are understandable. Most aspects of the literature review template are covered and completed though perhaps falling short of a thorough and professional level with perhaps some incomplete or mixed up aspects of the template.
● The introduction and motivation are written at an ok standard along with a meta-data figure but some aspects could be clearer.
● The field challenges are identified and described ok but could be presented more clearly or have some errors.
● The scope is described ok but could be improved.
● The search methodology is implemented and described ok along with a screenshot from ConnectedPapers.com but not necessarily to a good standard.
● The quality of the literature classification and organization is done ok and presented at including a screenshot + functioning URL of the Survis web page but not necessarily to a good standard.
● The paper summaries are written and presented ok but could be improved.
● The references are complete, in alphabetical order by first-author surname, and a DOI is provided for each paper.
Grade 40-49. A literature review of the research is provided and most sections reflect some understanding. Most aspects of the literature review template are covered and completed though with some vague descriptions, missing details, incomplete, mix-ups, or errors.
● The introduction and motivation are vague, lack clarity, or are not thought out.
● The field challenges are identified and described but are vague, lack clarity, or are erroneous.
● The scope is described but is vague, lacks clarity, or contains some error.
● The search methodology is incomplete, lacks clarity, or contains errors.
● The quality of the literature classification and organization is weak, is not well motivated, lacks clarity or does not make sense.
● The paper summaries are written and presented but lack clarity or contain errors.
● Some of the references are incomplete or erroneous.
Grade 30-39. A literature review of the research is provided but reflects little understanding. Some aspects of the literature review template are covered and completed though with some vague descriptions, missing details, incomplete, mix-ups, or errors.
● The introduction and motivation are not clearly explained and may even contain errors.
● The field challenges are vague, lack clarity, or are erroneous.
● The scope is vague, lacks clarity, or contains some error.
● The search methodology is incomplete, lacks clarity, or contains errors.
● The quality of the literature classification and organization is poor, is not well motivated, lacks clarity or does not make sense.
● The paper summaries are poor, lack clarity or contain errors.
● The quality of the references is low.
Grade 20-29. A literature review of the research is attempted but reflects little understanding. Some aspects of the literature review template are covered though with some vague descriptions, missing parts, incomplete, mix-ups, or errors.
● The introduction and motivation are either missing, not clearly explained, or contain errors.
● The field challenges are either missing, lack clarity, or are erroneous.
● The scope is either missing, is low quality, or contains errors.
● The search methodology is incomplete, missing, or contains errors.
● The quality of the literature classification and organization is either missing, poor, or does not make sense.
● The paper summaries are incomplete, poor, lack clarity, or contain errors.
● The references are incomplete or low quality.
Grade 10-19. No serious attempt at a literature review is attempted. There are some vague descriptions, missing parts, incomplete, mix-ups, or errors.
● The introduction and motivation are not seriously attempted..
● The field challenges are not seriously attempted.
● The scope is not attempted seriously.
● The search methodology is not seriously attempted..
● The literature classification and organization is not given a good attempt.
● The paper summaries are not seriously attempted.
● The references are incomplete or low quality.
Grade 0-9. No or minimal attempt
2023-08-31
(Abridged) Literature Review