Send the shutdown message to threads and do the thread shutdown functions before more forcefully sending pthread_cancel to threads.