While the re-factor or re-build is an engineering problem, you have to keep in mind the associated business problem. The answer should be based on a few fundamental questions you ask yourself.
Why? To fix Design or code problem related to function or performance ?
Design problem will probably lead to rebuild, figure it out early, not mid air.
What? To fix underlying Data structure, Core logic or UI / UX ?
Hope your application architecture is modular enough to allow changes in each layer independently. Refactor if you CAN Rebuild if you MUST
It takes a lot of users actively using a system for it to mature
A rebuild pushes all the mature code out the window.
Who? Is the change necessary for the User or yourself
If the change is necessary for yourself then the user should not even know that you switched out the entire underlying hardware and software.
If the change is to address user complaints then consult with the major users and make sure you include all the necessary changes before the trek.
Of course, this is assuming you have paying users that you do no want to lose or impact negatively.
If your system architecture allows making changes to the three layers
UI/UX, Logic and Data independent of each other then - REFACTOR
If it does not and you will lose customers unless you do something then REBUILD, do not procrastinate.