This assignment specification may be updated to reflect clarifications and modifications after it is first issued. -It is strongly suggested that you start working on the assignment right away. You can submit  as many times as you like. Only the most recent submission at the due date will be assessed.

In this assignment, you are required to submit a single report in the form of a PDF file.  You may also  attach supporting information (appendices) as one or more identified sections at the end of the same PDF file. Appendices will not be marked but may be treated as supporting information to your report. Please use a cover sheet at the front that identifies you as author of the work using your u-number    and name and identifies this as your submission for COMP3425 Assignment 2. The cover sheet and     appendices do not contribute to the page limit.

You are expected to write in a style appropriate to a professional report. You may refer to http://www.anu.edu.au/students/learningdevelopment/writing-ssessment/reporat-writingfor some stylistic advice. You are expected to use the question and sub-question numbering in this assignment to identify the relevant answers in your report.

No particular layout is specified, but you should use no smaller than 11-point typeface and stay      within the maximum specified page count. Page margins, heading sizes, paragraph breaks and so   forth are not specified but a professional style must be maintained. Text beyond the page limit will be treated as non-existent.

This is a single-person assignment and should be completed on your own.  Make certain you carefully reference  all  the  material  that  you  use,  although  the  nature  of  this  assignment  suggests  few references will be needed.  It is unacceptable to cut and paste another author's work and pass it off as  your  own. Anyone  found  doing  this,  from  whatever  source,  will  get  a  mark  of  zero  for the assignment and, in addition, CECC procedures for plagiarism will apply.

(b) Comment on whether, in general, association mining could be a useful technique on this data.

4. Study a very simple classification task

Aim to build a model to classify binary undecided voter.  Use binary undecided voter as the target    class and set every other variable (except srcid) as Input (independent).  Using sensible defaults for model parameters is fine for this exercise where we aim to compare methods rather than optimise them.

(a) This should be a very easy task for a learner. Why? Hint: Think how binary undecided voter is

defined.

(b) Train each of a Linear, Decision tree, SVM and Neural Net classifier, so you have 4 classifiers. Hint: Because the dataset is large, begin with a small training set, 20%, and where run-time           speeds are acceptable, move up to a 70% training set. Evaluate each of these 4 classifiers, using a confusion matrix and interpreting the results in the context of the learning task.

(c) Inspect the models themselves where that is possible to assist in your evaluation and to explain the performance results.  Which learner(s) performed best and why?

5. Predict a Numeric Variable

Q15_safe_gambler has been derived from a range of gambling behaviour questions in the survey      (see accompanying Information about the data). You are to train a regression tree or a neural net to predict Q15_safe_gambler, you may use any other variables as input.

(a) Explain which you chose of a regression tree or neural net and justify your choice.

(b) Train your chosen model and tune by setting controllable parameters to achieve a reasonable performance. Explain what parameters you varied and how, and the values you chose finally.

(c) Assess the performance of your best result using the subjective and objective evaluation appropriate for the method you chose and justify why you settled with that result.

6. More Complex Classification

Q4 of the survey asks respondents which political party they would vote for if an election were held now. Your task is to classify a person according to whether they are an undecided voter or not. Hint: The variable binary undecided voter has transformed the values of Q4 to a binary variable with values TRUE or FALSE, so you can use binary undecided_voter as your target. Hint: Be sure to ignore variable Q4 when binary undecided_voter is your target. Hint: Initially, use a small training set, 20%, and where run-time speeds are acceptable, experiment with a larger training set.

(a) Explain how you will partition the available dataset to train and validate classification models in (b) to (d) below.

(b) Train a Decision Tree Classifier. You will need to adjust default parameters to obtain optimal

performance. State what parameters you varied and (briefly) their effect on your results.

Evaluate your optimal classifier using the error matrix, ROC, and any quality information specific to the classifier method.

(c) Train an SVM Classifier. Then proceed as for (b) Decision Tree above, using your SVM classifier instead.

(d) Train a Neural Net classifier. Then proceed as for (b) Decision Tree above, using your Neural Net classifier instead.

7. Clustering

(a) Restore the dataset to its original distributed form, removing any new variables you may have constructed above.

For clustering, use the 3 raw variables, Q1, p_age_group_sdc and p_education_sdc, plus 2 variables of your choice from Q5a to Q5m (to total 5 variables).  To use the Q5 variables, ensure you choose <DUM1> (see question DUM1) to be one of ‘”1” for Universities in Australia or “2” for the Australian National University, but not both (that is, you will need to remove about half of the data rows).         Ignore all the other variables.

Rescale the variables to fall in the range [0-1] prior to clustering.  Use the full dataset for clustering (i.e. do not partition into train-test sets).

Experiment with clustering using the k-means algorithm by building cluster models for each of k= 2,

5, √ (the latter is a recommended default for dataset of size n) clusters. Choose your preferred k

and its cluster model for k-means to answer the following.

(a) Justify your choice of k as your preferred (Hint: have look at parts b-d below for each cluster model).

(b) Calculate the sum of the within-cluster-sum-of-squares for your chosen model.  The within- cluster-sum-of-squares is the sum of the squares of the Euclidean distance of each object from its cluster mean. Discuss why this is interesting.

(c) Look at the cluster centres for each variable.  Using this information, discuss qualitatively how each cluster differs from the others.

(d) Use a scatterplot to plot (a sample of) the objects projected on to each combination of 2 variables with objects mapped to each cluster by colour (Hint: The Data button on Rattle’s Cluster tab can do   this). Describe what you can see as the major influences on clustering. Include the image in your      answer.

8. Qualitative Summary of Findings (Hint: approx 1/2 page)

Comparatively evaluate the techniques you have used and their suitability or not for mining this       data. This should be a qualitative opinion that draws on what you have found already doing the        exercises above.  For example, what can you say about training and classification speeds, the size or other aspects of the training data, or the predictive power of the models built?  Finally, what else     would you propose to investigate as a follow-up to your work presented here?