In a large dutch oven, render the fat from the bacon over low heat. Once there's a good amount of fat in the pan, turn the heat up slightly and cook the bacon until crisp. Remove it to drain on paper towels. Discard all but approximately 2-3 tablespoons of the bacon fat.
Add the chopped onion, garlic and celery to the pan with a pinch of salt and sweat the vegetables for several minutes on medium-low heat. Once they're soft, add the cauliflower and turn up the heat to medium. Cook the vegetables for a few more minutes, stirring periodically, until the cauliflower has softened up. Add the chicken stock and thyme. Bring the mixture to a simmer, cover the pot and lower the heat. Allow the soup to simmer on low heat for 45 minutes.
Remove the thyme stems and carefully ladle the soup into a blender. Puree until silky. Return the soup to the pot and add the milk. Season with salt and pepper to taste.
Reheat the reserved bacon briefly, dice finely and use it to garnish each bowl of soup.