• Sebastian Thrun’s Stanford students built an ML model to identify cancer in images of skin.
• Survival rates drastically impacted by early detection.
• Hard for humans to do.
• Much harder than dog breed identification.
• Dermatologists are paid handsomely for their abilitiy to do it.

## Process

• First, build classification tree.
• Has over 2000 classifications
• 3 branches from root (Benign, malignant, non-neoplastic)
• Gather training data.
• Correctly labeled images of skin disease.
• Clean data by correcting for
• resolution
• lighting
• presence of foreign objects
• duplicates
• Build net
• Used stock RCNN
• Using net pretrained to identify animals actually improved skin cancer detection.
• Makes sense to me. There are some common characteristics between them: Outlines, shape, contrast.
• Train net to classify 757 outcomes
• Validate results
• Compare network success vs actual dermatologist success on same data set.

## Sensitivity/Specificity & Precision/Recall

• They’re all metrics of success
• Sensitivity:
• Of all the positive outcomes (In this case, the images that contained skin cancer), how many were labeled as positive?
• Specificity:
• Of all the negative outcomes (Or, the images that did not contain skin cancer), how many were labeled as negative?
• Recall:
• Synonymous with sensitivity. How many positive cases were correctly labeled by the network?
• Precision:
• Out of all the cases we labeled as positive, how many were actually positive?
• Where:
• True positives (TP)
• True negatives (TN)
• False positives (FP)
• False negatives (FN)

## Threshold for labeling cancer

• Set a low threshold for determining cancer
• It’s better to send a healthy person to the doctor for more tests than it is to tell a sick person they’re fine.

## ROC curves

• Metric to determine accuracy of a ‘split’
• perfect $1.0$
• good $0.8$
• bad (or random) $0.5$
• Requires finding
• True positive rate
• False positive rate
• For positive and negative labels
• Make coordinates out of $\frac{TP}{FP}$
• Plot them on a $0,0$ to $1,1$ graph
• Area under the curve is the final score

## How does the model know?

• We can’t say for sure, but these are visualizations of the feature maps that the network thinks are significant:

## Confusion Matrices

• A graph that says, ‘If I am class B, what are the chances of the NN classifying me as A or C?’