When Jack comes in, he confirms that he did have a fling with Sam