Excellent question, Nishant, and one I can speak to from experience.
On the one hand: ANYONE starting a technology company today should think globally from day one. Technology is global - period. Any time I put together a financial model or business plan for a new venture, the model is global and assumes >50% of revenue from outside the US (AT SOME POINT IN TIME).
On the other hand: when you are a bare metal startup, you are 100% focused on finding product-market-fit so you can book revenue and pay salaries. With bankruptcy constantly snapping at your heels, you tend to make some sacrifices on product - and globalization is an easy sacrifice to accept. This applies not only to American companies but also to UK companies, Japanese companies, Chinese companies, etc.
Your first and foremost concern as a entrepreneur is to make your company self-sustaining. If your local market can produce enough revenue to support your company's growth for 5 years, it's easy to just focus on that. On top that, investors and advisors will lobby for "laser focus". So even if you are inclined to "think global", you will face a lot of resistance to spending money, time, and energy on marketing overseas until you have already built a dominant business in your home market.
When companies take the "start local" approach, their ability to translate that success into a global phenomenon is directly proportional to their barriers to entry. Not to pick on any one business, but if your idea is to start an app that connects dog owners to dog walkers then the barriers to entry are quite low. You should expect that while you battle for dominance in your home country someone else is also achieving dominance in their home country. So, if you subsequently want to be the Global Leader in Dog Walking Apps, your path will be through acquisition as a roll-up play of many different properties. Conversely, if your app has some seriously difficult "secret sauce", you may have an opportunity to get traction in your home market, then take your unique technology to other countries with different languages and customs.
Interestingly, there are some in-between examples - like Uber. No offense, Uber, but the actual technology involved in connecting riders to drivers is NOT rocket science. However, there is a large marketing and investment barrier in terms of recruiting enough drivers and riders to achieve critical mass in any one geography. Uber has done a brilliant job of proving the model in one geography then raising enough money to quickly colonize many new geographies. This is a great example of global thinking from day one, with deep pocket investors who can support global aspirations.