Sorting II (incl. Insertion Sort, Shell's Sort, Shuffling)
Author: Josh Hug

Overview

Sorting. Why do we care about sorting so much? Useful as a subroutine for many other problems. Large number of interesting tradeoffs that we can use as exemplars to develop the more general skill of comparing algorithms.

Insertion Sort. For each item, insert into the output sequence in the appropriate place. Naive solution involves creation of a separate data structure. Memory efficient version swaps items one-by-one towards the left until they land in the right place. The invariant for this type of insertion sort is that every item to the left of position i is in sorted order, and everything to the right has not yet been examined. Every swap fixes exactly one inversion

Insertion Sort Runtime. In the best case, insertion sort takes θ(N) time. In the worst case, θ(N^2) time. More generally, runtime is no worse than the number of inversions.

Insertion Sort Sweet Spots. Insertion sort is very fast for arrays that are almost sorted, i.e. that have θ(N) inversions. It is also fastest for small N (roughly N < 15).

Shell's Sort. Not covered on exam. Idea is to compare items that are a distance h apart from each other, starting from large h and reducing down to h=1. The last step where h=1 ensures that the array is sorted (since h=1 is just insertion sort). The earlier steps help speed things up by making long distance moves, fixing many inversions at once. Theoretical analysis of Shell's sort is highly technical and has surprising results.

Sorting Implementations. Be comfortable with reading the array based sorting code provided. You are not expected to know the code for these sorting algorithms by heart (though trying to implement them is not a bad idea, especially as you approach job interviews).

Insertion Sort as a Subroutine. Since insertion sortis fast for arrays of size N < 15, it is best touse insertion sort for small subarrays in recursive algorithms.

Shuffling. One way to shuffle an array: Attach a random number to each item. Then sort. Resulting order will be totally random. Faster shuffling approaches exist.

B level

  1. Give a best and worst case input for insertion sort.
  2. Which sort do you expect to run more quickly on a reversed array, selection sort or insertion sort?

See sorting comparisons (lec30) guide for more questions that involve all of our sorting algorithms.