Trouble emerges once banks start to engage in lending unbacked by real savings, this gives rise to the expansion of credit out of “thin air.” This in turn sets in motion the menace of the boom-bust cycle.
The assertion that “tax-financed public goods can make us all better off” is just that: an assertion. As Rothbard showed, there is no reason to just assume consumers would pay for these amenities were they not forced to through taxation.
It makes sense that individual ethics can vary within a natural law framework: the nonaggression principle isn't so much a rule to be applied as a norm to be followed. It's the foundation that makes life among other people possible.