Aim

The aim of this assignment is to investigate and visualise data using Python as a data science tool. It will test your ability to:

1.   read a data file in Python and extract related data from it.

2.   use various graphical  and non-graphical tools for performing exploratory data analysis and visualisation.

3.   use basic tools for managing and processing data and

4.   communicate your findings in your report/video recording.

Data

The data we will use contains the number of monthly smartcard replacements by reason and type in Queensland and comes from the Queensland government open data initiative.

•     The   monthly   smartcard   replacements   dataset   (monthly_smartcard_replacements.csv) contains all recorded smartcard replacements in Queensland for different smartcard types and reasons each month.

•     The information is given under variables; Month (including year and month), Transaction, Smartcard.Type, Action.Reason and Number.of.transactions.

•     The file (monthly_smartcard_replacements.csv) is available on the unit Moodle site under Assessments.

Hand-in Requirements

Please  hand  in  a  PDF file  containing your code, answers and explanations to questions and a Jupyter notebook file (.ipynb) containing your Python code to all the questions respectively:

●    The PDF file should contain:

o    Answers     and    explanations     to     the    questions.     Make    sure     to    include screenshots/images of the graphs you generat. Also, copy/paste your Python code to justify your answers for all the questions.

o    You  can  use  Microsoft  Word  or  other word  processing  software  to  format  your submission. Alternatively, generate your PDF from your jupyter notebook formatted using markdown. Either way save the final copy to a PDF before submitting.

●    The .ipynb file should contain:

o    A copy of your work using python code to answer all the questions.

●    The video file should contain:

o    A recording of yourself, explaining your answers to a subtask from Task A.

o    You can use Zoom to prepare your recording.

o    Note each student is required to explain only one subtask from Task A. Please see Task B for more details.

You will need to submit three separate files (i.e., .pdf file, .ipynb file and your video file). Zip, rar or any other similar file compression format is not acceptable and will have a penalty of 10%.

Assignment Tasks:

Note: You need to use Python to complete all tasks.

Task A: Data Exploration and Visualisation

In this task, you are required to explore the monthly smartcard replacements dataset and  perform  analysis  based  on  data  subsets  or  groups  with  visualisations  where required. Read the CSV file (monthly_smartcard_replacements.csv) in Python and then answer a series of questions about the data using Python.

A1. Exploring Smartcard Types

1.  How    many    different     (unique)    smartcard    types    are    recorded     in    the ‘Smartcard.Type’  column?  What  are  those  different  smartcard types and  how many instances are recorded for each type?

2.  Plot a barchart of the smartcard types with the bars showing the count of each smartcard type. Which smartcard type is replaced the most?

3.  Calculate the percentage of records for each smartcard type.

A2. Exploring Reasons for Smartcard Replacement

1.  Convert data type of column ‘Month’ to a datetime format.

Hint: Use pandas.to_datetime function to convert the type of ‘Month’ column to a datetime format as shown in one of your applied sessions.

2.  What are the different reasons for smartcard replacements in the given data and how   many    instances   are   observed   for   each   reason?    Hint:   Check   the 'Action.Reason' column.

3.  What   is   the   total    number   of   months   in   which   100   or   more   smartcard replacements are reported due to being "Faulty"?

A3. Investigating Annual Smartcard Replacements

1.  Create a new column named 'Year'extracting the year from the 'Month' column.   Hint: you can extract year from column ‘Month’ using method .dt.year and create a new column for year as follows:

>>> your_dataframe['Year']=your_dataframe['Month'].dt.year

2.  Create a line plot showing the total number of annual smartcard replacements (number of transactions) against year.

3.  Explain the trend  as  observed  from  the  chart.  Are  there  any years that are different from others and if so, what is the reason behind it?