Thank you for sharing your experience and thoughts Josh.
For what it's worth, few people can afford to pay off a mortgage in two years. Generally, it's if you have a lot of money available on hand relative to the cost of the home you buy, and/or your income is far greater than your expenses.
Regardless, I'd argue that if you had invested that money prudently, used the tax deduction for the interest, and simply allowed inflation to slowly chip away at the value of the dollars you owed, you would have come out much further ahead, even counting the interest payments.
As for the safety of not having a mortgage, you could have kept investing the money you used to pay off the mortgage, which would have allowed you to pay it off in full at will.
Despite all that, whatever works best for you is all that matters in your personal situation.